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FOREWORD 


This  is  Volume  II  of  the  final  report  of  Task  Group  II  of  the  Weapon 
System  Effectiveness  Industry  Advisory  Committee  (WSEIAC).  It  is  sub¬ 
mitted  to  the  Commander,  AFSC,  in  partial  fulfillment  of  Ta^u:  Group  II 
objectives  cited  in  the  committee  Charter.  The  final  report  is  contained  in 
three  separate  volumes: 

Volume  I  contains  an  overview  of  Task  Group  II  findings, 
including  a  summary  of  Volumes  II  and  III,  conclusions, 
a.nd  recommendations. 


Volume  II  contains  a  discussion  of  effectiveness  concepts, 
a  description  of  specific  tasks  required  to  evaluate  effec¬ 
tiveness,  and  a  detailed  example  illustrating  the  method. 

Volume  III  contains  descriptions  of  effectiveness  analysis 
methods  applied  to  four  typical  Air  Force  systems  using 
the  techniques  described  in  Volume  II. 

The  membership  of  Task  Group  11  was  as  follows: 


Mr.  D.  F.  Barber  (Chairman) 

Mr.  I.  Bosinoff 

Mr.  I.  Doshay 
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Mr.  C.  R.  Knight  (Technical 
Director) 
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Mr.  M.  M.  Tall 
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ARINC  Research  Corporation 
ARINC  Research  Corporation 

TRW  Space  Technology  Laboratories 
00AM A  (OONEW) 

Radio  Corporation  of  America 
Hughes  Aircraft  Company 


Other  task  group  reports  submitted  in  fulfillment  of  the  committee's 
objectives  are: 

AFSC-TR-65-1  Final  Report  of  Task  Group  I 
"Requirements  Methodology" 

AFSC-TR-65-3  Final  Report  of  Task  Group  III 

"Data  Collection  and  Management  Reports" 
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AFSC-TR-65-4  Final  Report  of  Task  Group  IV 

"Cost-Effectiveness  Optimization" 

AFSC-TR-65-5  Final  Report  of  Task  Group  V 
"Management  Systems" 

AFSC-TR“65-6  Final  Summary  Report 

"Chairman's  Final  Report" 

Publication  of  this  report  does  not  constitute  Air  Force  Approval  of  the 
report's  findings  or  conclusions.  It  is  published  only  for  the  exchange  and 
stimulation  of  ideas. 

APPROVED 


William  F.  Stevens,  Colonel,  USAF 
Chief,  Systems  Effectiveness  Division 
Directorate  of  Systems  Policy 
DCS  Systems 


i~n~nnwriTirTifri  iiiiii  iin  '  iibaagiTBiiBitarrnfiriiMriiWTirwTiiTTT? 


.v.'HM««(Uavatmr 


WSEIAC  CHARTiJl 


In  order  that  this  report  of  Task  Group  11  be  );L\jd.'ed  in  context  with 
the  entire  committee  effort,  the  purpose  and  task  group  objectives  as  stated 
in  the  WSEIAC  Charter  are  listed  below: 

Purpose 

The  purpose  of  the  Weapon  System  Effectiveness  Industry  Advisory 
Committee  is  to  provide  technical  guidance  and  assistance  to  AFSC  in  the 
development  of  a  technique  to  apprise  management  of  current  and  predicted 
weapon  system  effectiveness  at  all  phases  of  weapon  system  life. 

Task  Group  Objectives 

Task  Group  1  -  Review  present  procedures  being  used  to  establish  system 
effectiveness  requirements  and  recommend  a  method  for  arriving  at  require¬ 
ments  that  are  mission  responsive. 

Task  Group  II  -  Review  existing  documents  and  recommend  uniform  methods 
and  procedures  to  be  applied  in  predicting  and  measuring  systems  effective- 
ness  during  all  phases  of  a  weapon  system  program. 

Task  Group  III  -  Review  format  and  engineering  data  content  of  existing 
system  effectiveness  reports  and  recommend  uniform  procedures  for 
periodically  reporting  weapon  system  status  to  assist  all  levels  of  manage¬ 
ment  in  arriving  at  program  decisions. 

Task  Group  IV  -  Develop  a  basic  set  of  instructions  and  procedures  for 
conducting  an  analysis  for  system  optimization  considering  effectiveness, 
time  schedules,  and  funding. 

Task  Group  V  -  Review  current  policies  and  procedures  of  other  Air  Force 
commands  and  develop  a  framework  for  standardizing  management  visibility 
procedures  throughout  all  Air  Force  commands. 


ABSTRACT 


Concepts  of  system  effectiveness  including  the  three  principal  terms, 
availability,  dependability,  and  capability,  are  presented.  Eight  specific 
tasks  required  to  evaluate  effectiveness  during  any  phase  of  system  life  are 
presented.  A  mathematical  routine  appropriate  to  effectiveness  model  con¬ 
struction  is  described.  Using  the  above  task  analysis  and  the  model  frame¬ 
work,  a  hypothetical  example  is  presented.  Results  of  the  evaluation 
illustrate  effectiveness  analysis  methods  and  possible  alternate  decisions 
available.  Application  of  simulation  methods  to  the  example  are  discussed. 
The  appendixes  contain  summaries  of  four  typical  examples  of  the  applica¬ 
tion  of  effectiveness  evaluation  methods  to  various  Air  Force  systems 
(presented  in  detail  in  Volume  III).  An  airborne  avionics  system,  an  inter¬ 
continental  ballistic  missile  system,  a  long  range  radar  surveillance  system, 
and  a  spacecraft  system  are  described. 
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SECTION  I 


INTRODUCTION 

The  design  and  development  of  military  systems  has  traditionally 
crowded  the  state  of  the  art  in  materials,  devices,  and  physical 
principles.  In  recent  times,  designers  have  been  faced  simultaneously 
with  increasingly  novel  demands  and  ever  more  acutely  limited  test  data. 
Performance  requirements  invariably  include  severe  reaction  and  response 
time  limits  which  cannot  be  met  without  a  close  integration  of  personnel, 
procedures,  and  hardware.  At  the  same  time  program  cost  reductions, 
accelerated  development  schedules,  and  lack  of  opportunity  for  complete 
system  tests  prior  to  operational  deployment  have  combined  to  reduce  the 
opportunity  to  obtain  extensive  operational  usage  data,  either  in  kind  or 
quantity.  Accordingly,  what  was  once  considered  merely  desirable  is  now 
mandatory  --an  integrated  methodology  of  system  program  management 
utilizing  all  available  data  both  to  pinpoint  problem  areas  and  to  provide  a 
numerical  estimate  of  system  effectiveness  during  all  phases  of  the  system 
life  cycle. 

It  is  the  specific  objective  of  Task  Group  II  to  "recommend  methods 
and  procedures  for  measuring  and  predicting  system  effectiveness  during 
all  phases  of  a  program.  "  Mathematical  models  utilizing  analytical  methods 
and  maciiine  simulation  programs  are  an  essential  part  of  an  integrated 
methodology. 

Task  Group  II  adopted  a  framework  for  system  effectiveness  evaluation 
based  on  three  factors: 

(1)  availability  (readiness) 

{Z)  dependability  (reliability) 

(3)  capability  (performance) 

This  framework  was  organized  into  a  specific  analytical  structure, 
the  use  of  which  is  illustrated  in  several  examples. 

The  report  discusses  the  general  concepts  associated  with  system 
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effectiveness.  This  discussion  is  followed  by  a  description  of  the  tasks 
which  must  be  performed  in  order  to  arrive  at  a  numerical  estimate  of 
system  effectiveness,  and  to  obtain  insight  into  the  controllable  factors  of 
the  system  that  influence  effectiveness.  Finally,  a  tutorial  example  is 
given,  illustrating  methods  of  analysis  in  the  formal  analytical  framework 
adopted  by  Task  Group  11. 
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GENERAL  CONCEPTS 

This  section  introduces  concepts  of  system  effectiveness  evaluation. 
Specific  definitions  as  they  are  employed  in  model  oonatruction  are  pre¬ 
sented  in  Section  IV. 

System  Effectiveness  (Reference  1)  is  a  measure  of  the  extent  to 
which  a  system  may  be  expected  to  achieve  a  set  of  specific  mission 
requirements  and  is  a  function  of  availability,  dependability,  and 
capability. 

Availability  is  a  meas^’.re  of  the  system  condition  at  the  start  of  a 
mission  and  is  a  function  of  the  relationships  among  hardware, 
personnel,  and  procedures. 

Dependability  is  a  measure  of  the  system  condition  at  one  or  more 
points  during  the  mission;  given  the  system  condition(s)  at  the  start 
of  the  mission  and  may  be  stated  as  the  probability  (or  probabilities 
or  other  suitable  mission  oriented  measure)  that  the  system  (1)  will 
enter  and/or  occupy  any  one  of  its  significant  states  during  a  speci¬ 
fied  mission  and,  (Z)  will  perform  the  functions  associated  with  those 
states. 

Capability  is  a  measure  of  the  ability  of  a  system  to  achieve  the  mis¬ 
sion  objectives;  given  the  system  condition(s)  during  the  mission,  and 
specifically  accounts  for  the  performance  spectrum  of  a  system. 

The  objectives  of  system  effectiveness  evaluation  are  to: 

(1)  Evaluate  system  designs  and  compare  alternative  configurations. 

(2)  Provide  numerical  estimates  for  use  in  defense  planning. 

(3)  Provide  management  visibility  at  every  phase  of  system's  life 
cycle  of  the  extent  to  which  the  system  is  expected  to  meet  its 
operational  requirements  (SOR). 

(4)  Provide  timely  indication  of  the  necessity  for  corrective  actions. 

(5)  Compare  the  effect  of  alternative  corrective  actions. 
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SECTION  m 


DESCRIPTION  OF  TASKS  REQUIRED  TO 
EVALUATE  SYSTEM  EFFECTIVENESS 

3 .  1  Introduction 

This  section  of  the  report  describes  eight  tasks  that  must  be 
performed  in  evaluating  effectiveness.  These  tasks  are  discussed  in 
terms  of  the  Requirements  and  available  information  during  tlie  four 
phases  of  system  life:  (1)  conceptual;  (2)  definition;  (3)  acquisition;  and 
(4)  operational. 

3. 2  Phases  of  System  Life 

The  objectives  of  each  of  the  four  phases  of  system  life  are  described 
below. 

3.2.1  Conceptual  Phase 

The  objectives  are  to  establish  a  feasible  technical  approach 
for  satisfying  a  given  requirement;  to  evaluate  whether  the  approach  is 
worth  pursuing  or  whether  the  military  requirement  should  be  satisfied 
in  another  manner.  The  phase  extends  from  determination  of  a  broad 
objective  or  need  to  Air  Force  approval  of  the  Program  Change  Proposal 
covering  the  Definition  Phase. 

3.2.2  Definition  Phase 

The  objectives  are  to  select  and  define  the  specific  system 
configuration,  to  establish  performance  specifications,  to  provide  cost 
and  schedule  estimates  and  to  confirm  the  desirability  of  acquiring  the  sys¬ 
tem  for  use.  This  phase  is  initiated  by  System  Definition  Directive  and  ends 
with  issuance  of  a  System  Program  Directive. 

3.2.3  Acquisition  Phase 

The  objectives  are  to  carry  out  detailed  design  and  development, 
conduct  category  tests,  and  procure  required  quantities  of  hardware.  The 
period  starts  after  issuance  of  the  System  Program  Directive  and  ends  with 
acceptance  by  the  user  of  the  last  operating  unit  in  a  certain  series  or  until 


the  SOR  has  been  demonstrated  through  Category  T.  testing  and  all  required 
updating  changes  resulting  from  the  testing  have  bten  identified,  approved, 
and  placed  on  procurement,  whichever  occurs  latei . 
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3.2.4  Operational  Phase 

The  objective  is  to  employ  the  procured  system  in  an  effective 
manner.  This  phase  begins  with  acceptance  by  the  user  of  the  first 
operating  unit  and  continues  until  final  disposition  of  the  system.  It  overlaps 
the  Acquisition  Phase. 

Figure  1,  extracted  from  AFR  375- 1,  '  Management  of 
System  Programs,  ”  shows  the  four  phases  in  terms  of  the  Air  Force 
decision  process. 

3.  3  Tasks 

Eight  tasks  used  in  evaluating  effectiveness  are  listed  below: 

(1)  Mission  definition 

(2)  System  description 

(3)  Specification  of  figure(s)  of  merit 

(4)  Identification  of  accountable  factors 

(5)  Model  construction 

(6)  Data  acquisition 

(7)  Parameter  estimation 

(8)  Model  exercise. 

The  tasks  are  described  below  and  are  followed  by  a  discussion  of  how 
they  relate  to  the  four  phases  of  system  life. 

Figure  2  shows  the  eight  tasks  leading  to  an  evaluation  of  effective¬ 
ness. 

3.3,1  Mission  Definition 

The  mission  definition  is  a  precise  statement  of  the  intended 
purpose(s)  of  the  system  and  of  the  environmental  conditions  (natural  and 
man-made)  under  which  it  is  required  to  operate. 
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Category  HI  tests  (AFR  80-14) 
Continued  support  of  operational 


- Iterative  process 


FIGURE  2.  PRINCIP4L  TASKS  REQUIRED  FOR 

EVALUATION  OF  SYSTEM  EFFECTIVENESS 
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3.3.Z  System  Description 

3.3. Z.1  General  Configuration 

The  major  hardware  components  of  the  system  must  be 
described  and  their  functions  defined. 

3.3. Z.Z  System  Block  Diagram 

A  block  diagram  of  the  system  should  be  constructed 
showing  signal  flow  and  redundancy  (Reference  Z). 

3.3. Z.3  Mission  Profile 

A  time-line  analysis  showing  the  sequence  of  events 
from  initiation  of  each  mission  to  its  completion  should  be  prepared.  This 
delineation  may  split  the  mission  into  a  number  of  discrete  time  intervals 
during  which  different  functions  are  being  performed.  The  components 
used  in  each  of  these  intervals  must  be  identified,  along  with  their  contri¬ 
bution  to  mission  success. 


3.3.Z.4  Mission  Outcomes 

The  principal  events  that  might  result  from  a  mission 
must  be  selected  and  differentiated.  In  some  cases,  it  is  sufficient  to 
differentiate  between  successful  fulfiilment  of  the  purpose  a->.d  failure  to 
fulfill  that  purpose.  In  other  cases,  there  are  possibilities  of  partial  suc¬ 
cess  or  even  continuous  gradations  of  outcomes  ranging  from  total  success 
to  total  failure. 

3.3.3  Specification  of  Figure(s)  of  Merit 

In  general,  a  figure  of  merit  is  any  index  which  indicates  the 
quality  of  a  system.  In  the  simplest  case  it  may  be  a  measured  physical 
quantity,  such  as  range  or  payload.  On  the  other  hand,  it  may  be  a  calcu¬ 
lated  quantity  based  on  measurement,  such  as  mean  down  time  or  mean 
time  between  maintenance  actions.  Lastly,  it  may  be  a  predicted  quantity 
based  on  measurement  and/or  simulation.  For  example,  "the  probability 
that  a  system  can  meet  an  operational  demand  at  a  random  point  in  time 
while  under  attack,  "  will  require  prediction  since  there  will  be  some 
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uncertainty  about  the  attack  enWronment- 

Figure(s)  of  Merit  serve  to  indicate  what  can  be  expected  from 
the  system.  They  must  be  in  an  operationally-oriented  form  that  can  be 
readily  understood  and  utilized  in  planning  (References  3  and  4).  Where  the 
number  of  significantly  different  mission  outcomes  is  small,  the  probabili¬ 
ties  of  each  of  these  outcomes  can  be  the  appropriate  figures  of  merit. 

When  the  number  of  mission  outcomes  is  large  or  when  a  continuous  range 
of  outcomes  requires  consideration,  a  measure  of  relative  "adequacy"  may 
be  assigned  to  each  possible  outcome,  and  the  expected  "adequacy"  should 
be  used  as  a  figure  of  merit. 

System  Effectiveness  is  defined  as  the  vector  of  the  specified 
figure(s)  of  merit. 

3.3.4  Identification  of  Accountable  Factors 

Accountable  factors  are  those  specific  factors  which  are  known 
or  suspected  to  have  a  significant  influence  on  the  figure(s)  of  merit.  All 
assumptions  which  are  made  in  regard  to  these  factors  must  be  explicitly 
stated.  Thus,  it  is  essential  to  preface  any  analysis  by  a  list  of  the  assump¬ 
tions  made  concerning  the  intrinsic  failure  and  repair  characteristics  of  the 
components  (e.g.,  exponential  distributions),  the  maintenance  policies  in 
effect  (e.g.,  preventive  maintenance  schedules,  checkout  procedures),  and 
the  environmental  conditions  under  which  the  system  is  to  operate  (e.g., 
temperature  extremes,  vibration,  enemy  countermeasures,  etc.).  Since 
the  relative  importance  of  a  specific  factor  is  a  strong  function  of  the  phase 
of  system  evolution,  a  periodic  review  of  this  listing  should  be  made  to 
ascertain  that  the  model  contains  at  least  those  factors  which  can  be  influ¬ 
enced  by  the  decision  maker  at  that  particular  point  in  the  system  life. 

Table  I  is  a  checklist  for  identification  of  accountable  factors. 

3,  3.  5  Model  Construction 

The  model  is  a  technique  for  combining  the  information 
developed  in  the  prior  four  tasks  in  order  to  estimate  system  effectiveness. 
The  model  serves  as  a  probabilistic  representation  of  the  events  which  may 
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occur  prior  to  and  during  a  mission.  It  relates  these  possible  events  to  the 
levels  of  performance  adequacy  which  may  be  expected  for  the  mission. 


TABLE  I 

CHECKLIST  FOR  IDENTIFICATION  OF  ACCOUNTABLE  FACTORS 


System  Hardware  Description 
.  Modes  of  operation 
.  Hardware  organization 

Compatibility 

(e.g.,  Electromagiietic 
Compatibility) 

Survivability 

Vulnerability 

Deployment 

Geographic  Factors 
.  Deployment 
.  Geology 
.  Climate 

I 

.  Atmospheric  phenomena 

Personnel 
.  Operating 
.  Maintenance 


Spares 

.  Provisioning 
.  Storage 
.  Packaging 

Transportation 

Support  Equipment 
.  Test 
.  Transport 
.  Maintenance 
.  Facilities 

Procedures/ Policies 
.  Operating 
.  Repail 

.  Inspection/Maintenance 
.  Testing 

System  Interfaces 

.  Support  systems 

.  Force  mix 

.  Strategic  Integrated 

Operations  Plan  (SIOP) 


■  i? 
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The  first  step  in  model  construction  is  to  describe  the  signifi= 
cantly  different  system  "states"  in  which  the  mission  may  be  carried  out. 
System  "states"  are  distinguishable  conditions  of  the  system  which  result 
from  events  occurring  prior  to  and  during  the  mission.  For  example,  the 
condition  in  which  all  system  hardware  is  functioning  within  design  specifi¬ 
cations  is  one  state.  The  condition  in  which  the  system  is  completely 
inoperable  due  to  hardware,  personnel,  or  procedures  is  a  state  at  the  other 
extreme.  The  conditions  of  partial  system  operation  due  to  defects  of  hard¬ 
ware,  personnel,  or  procedures  are  represented  by  the  intermediate  system 
states.  It  should  be  evident  that  the  system  can  make  transitions  from  state 
to  state  during  a  mission.  The  time-line  analysis  performed  in  accordance 
with  Section  3.  3.2.  3  above,  may  have  split  the  mission  into  a  number  of  dis¬ 
crete  time  intervals  during  which  different  functions  are  being  performed 
and  different  portions  of  the  system's  hardware  are  being  used.  For  each 
discrete  time  interval,  a  set  of  significant  states  appropriate  to  the  function 
being  performed  during  that  interval  may  be  defined. 


The  next  step  is  to  relate  to  accountable  factors  the  probabilities 
of  each  of  the  sets  of  significant  states  which  are  appropriate  at  the  beginning 
of  the  mission.  This  array  of  probabilities  is  called  the  availability  vector. 
For  each  succeeding  time  interval,  an  array  of  state  probabilities  is  related 
to  accountable  factors.  These  probabilities  are  dependent  or  conditional  on 
the  effective  state  during  the  previous  time  interval.  For  example,  where 
no  repair  is  possible,  a  failure  in  one  interval  predetermines  the  possible 
states  in  the  succeeding  intervals.  These  arrays  of  conditional  probabilities 
are  called  the  dependability  matrices. 


A  simplified  method  of  analysis  which  is  generally  employed, 
defines  the  significantly  different  system  states  over  the  entire  mission 
rather  than  for  each  discrete  time  interval.  The  array  of  state  probabilities 
at  the  beginning  of  the  mission  still  yields  the  availability  vector.  However, 
the  dependability  matrix  contains  the  probabilities  of  the  effective  states 
throughout  the  mission  conditional  on  the  initial  states. 


A: 
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For  the  simplified  method,  the  next  step  is  the  construction  of 
the  capability  matrix.  This  is  an  array  of  numbers  which  are  a  measure  of 
the  ability  of  a  system  to  achieve  the  mission  objectives;  given  the  system 
condition{s)  during  the  mission.  This  array  of  numbers  (vector  or  matrix) 
specifically  accounts  for  the  performance  spectrum  of  the  system.  This 
occurs,  for  example,  when  the  accumulation  of  subsystem  performance 
deviations,  each  within  acceptable  tolerances,  results  in  a  bomb  drop  being 
wide  of  the  mark.  In  this  case,  there  has  been  no  specific  subsystem  mal¬ 
function,  but  a  system  malfunction  (or  performance  degradation)  due  to  the 
unlikely  combination  of  v/ithin  tolerance  variations  of  the  subsystem.  There 
may,  therefore,  be  a  continuous  spectrum  of  possible  mission  results,  none 
of  which  is  an  unequivocal  failure  or  success.  The  capability  matrix  repre¬ 
sents  the  expected  figures  of  merit  for  the  sysi'jm.  Each  element  of  the 
matrix  is  an  expected  figure  of  merit  conditional  on  carrying  out  the  mis¬ 
sion  in  a  given  effective  state.  The  matrix  has  a  column  for  each  figure  of 
merit  selected  in  accordance  with  Section  3.  3.  3  and  a  row  for  each  effective 
state. 

Defining  the  capability  matrices  for  each  of  the  time  intervals 
discussed  in  the  earlier  general  case  becomes  quite  complex.  It  is  for  this 
reason,  that  the  simplified  approach  is  used  so  generally.  A  discussion  on 
defining  and  combining  the  capability  matrices  for  the  general  case  will  be 
found  in  A  Model  Framework  for  System  Effectiveness.—^ 

Model  construction  has  been  described  in  four  steps:  (1)  state 
description;  (2)  determination  of  availability  vector;  (3)  determination  of 
dependability  matrix;  and,  (4)  determination  of  capability  matrix.  In  speci¬ 
fic  system  cases,  it  may  be  impractical  to  construct  the  model  exactly 
following  these  steps.  For  example,  it  maynot  be  desirable  cr  practical  to 
separate  the  dependability  and  capability  matrices  in  some  instances.  How¬ 
ever,  the  four  steps  do  serve  as  a  useful  guide  in  constructing  the  model. 

1/  See  Volume  III  (Technical  Supplement)  A  Model  Framework  for  System 
Effectiveness,  H.  S.  Balaban,  ARINC  Research  Corp. ,  prepared  for 
WSEIACi,  Task  Group  IV. 
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The  available  informational  inputs  and  the  decisions  to  be 
made  on  the  basis  of  the  outputs  strongly  influence  the  structure  of  the  model. 
These  models  may  also  be  subdivided  by  level  of  system  evaluation  such  as 
the  overall  system,  subsystems,  equipments  and  module  (or  piece  part) 
levels.  The  level  of  evaluation  depends  on  the  objective  of  the  particular 
evaluation  and  the  available  informational  inputs.  The  steps  of  model  con¬ 
struction  are  described  in  mathematical  detail  in  Section  IV. 

3.3.6  Data  Acquisition 

The  accountable  factors  determined  in  Section  3.3.4  and  the 
model  detail  level  determined  in  Section  3.  3.  5  imply  data  element  require¬ 
ments.  These  must  be  specifically  identified  in  the  data  acquisition  task. 

The  specification  of  data  elements  is  a  two-way  proposition.  The  analyst 
can  answer  only  those  questions  for  which  there  is  an  "adequate"  data  base. 
"Adequate"  to  the  analyst  may  not  be  consistent  with  constraints  of  time, 
cost  or  schedules  imposed  upon  the  project  manager. 

The  source  of  data  elements  and  the  method  of  collection  (i.  e.  , 
from  standard  Air  Force  reporting  forms,  from  category  tests  performed 
by  a  contractor,  etc.  )  should  be  stated.  Data  may  be  obtained  from  pub¬ 
lished  reference  material  such  as  the  Inter  service  Data  Exchange  Program 
(IDEP),  Mil  Hdbk  217,  or  other  generic  data  source s^Ancluding  historic 
information  from  earlier  systems.  A  listing  of  typical  data  elements 
required  is  shown  in  Table  II—/  Typical  data  elements  are  time  to  failure 
and  repair  time  of  components  (Reference  5).  Care  should  be  taken  to 
ensure  that  all  parameters  used  in  models  have  an  available  data  source. 

The  completeness,  the  appropriateness,  and  the  compatibility 
of  available  sources  of  data  constitute  the  largest  cause  for  difference  in  the 
evaluation  of  effectiveness  from  one  phase  to  another  in  the  system  life. 

ils  ee  BIBLIOGRAPHY 
3  / 

—  Op.  Cit'.  ,  Task  Group  III  Final  Report 
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TABLE  II 


TYPICAL  DATA  ELEMENT  REQUIREMENTS 


(1)  General  Identification  information 
(nor^  enclaturo,  etc.) 

(Z)  Time  information  {chronological 
time  and  sequence  of  events). 

(a)  Operating  Times 

1)  mission  time  and  phases 
Z)  non-mission  time 

a)  checkout  and  test 
time 

b)  full  on  standby 

c)  partial  on  standby 

(b)  Non-operating  Time  s 

1)  off,  no  demand 

a)  storage 

b)  free  time 

Z)  downtime  (when  in  demand) 

a)  repiiir  time 

b)  logistic  time  (spares, 
transportation,  queuing, 
other  support-oriented 
items) 

c)  administrative  time 
(training,  other  cause 
of  personnel  non¬ 
availability) 

d)  effect  of  emergency 
procedures 

(3)  Event  information 

(a)  Failure  events 

1)  identification  of  failure 
Z)  effect  on  mission  capability 

a)  critical 

b)  non-critical 

3)  repairable  during  mission 

4)  how  detected 

5)  failure  cUl  sification 

a)  primary 

b)  secondary 

6)  cause  classification 

a)  design 

b)  operational  environment 
1)  controlled 

Z)  uncontrolled 

c)  personnel  induced 
1)  supplier 

Z)  user 

d)  time -dependent 


(b)  Maintenance  events 

1)  classes  of  maintenanee  (in¬ 
cludes  monitoring  and  system 

exercising) 

a)  corrective  maintenance 
1)  scheduled 

Z)  unscheduled 

b)  preventive  maintenance 
1)  scheduled 

Z)  unscheduled 
Z)  event  information 

a)  type  of  action 
1)  replacement 
Z)  adiustment 
3)  repairs 

a)  in  place 

b)  other  location 

b)  manhours  expended  (minimum 
number  of  personnel  required) 

c)  level  of  personnel 

d)  adequacy  of  equipment  and 
tools 

e)  availability  and  quality  of 
spares 

f)  adequacy  of  facilities 

g)  adequacy  of  technical  data 

h)  adequacy  of  maintenance 
action 

(4)  Capability  information 

(a)  Determine  that  the  system  complies 
with  specified  requirements 

(b)  Determine  these  significant  perfor¬ 
mance  parameters  which  contribute 
to  mission  success 

(c)  Based  on  performance  of  actual  mission 
and/or  inferred  performance  from 
system  simulations,  measure  the 
capability  of  the  system  to  accomplish 
mission  objectives  vdthin  a  stated 
environment,  and  when  the  system  per¬ 
forms  in  accordance  with  specifications. 
This  is  a  difficult  measurement  to 
obtain  on  many  systems.  Therefore, 
calculations  of  inferred  or  expected 
performance  will  be  required  from 
system  simulations. 

(d)  To  the  degree  possible,  measure  or 
calculate  the  effect  of  various  environ¬ 
ments  on  the  system  capability. 
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During  the  conceptual  phase,  heavy  reliance  must  be  placed  on  sources  of 
generic  data,  on  results  learned  from  similar  systems  and  on  application  of 
basic  knowledge  about  physical  laws  appropriate  to  the  system  concept.  Late 
in  the  development  period  and  during  operational  use,  a  great  deal  more  data 
appropriate  to  the  evaluation  can  be  made  available.  However,  testing  is 
expensive, and  present  data  retrieval  systems  are  not  well  suited  to  effec¬ 
tiveness  analysis.  These  factors  strongly  indicate  that  planning  for  effec¬ 
tiveness  evaluation  must  be  an  integral  and  carefully  identified  portion  of  all 
phases  of  the  system  life  cycle. 


3.3.7  Parameter  Estimation  (Reference  6) 

Processing  the  data  elements  to  derive  numerical  estimates  for 
the  parameters  of  the  model  is  the  next  task.  The  analytical  techniques 
used  to  reduce  the  data  are  referred  to  here  as  "parameter  estimation" 
techniques. 
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The  specific  methods  used  depend  upon: 

(1)  The  nature  of  the  quantity  being  estimated 

(2)  The  control  which  can  be  exerted  over  the 
physical  mechanisms  which  generate  the 
data 

(3)  The  format  of  data  collection. 


The  simple  case  that  occurs  when  a  control  population  is 
tested  at  one  environmental  stress  level  for  the  exponential  or  Weibull 
distributions  of  failures  has  been  extensively  treated  in  the  literature.  The 
problems  that  arise  when  the  data  are  fortuitously  collected  at  a  variety  of 
environmental  stress  levels  (as  is  the  usual  case  with  field-generated  data) 
have  been  much  less  thoroughly  investigated.  In  particular,  the  question 
of  accuracy  of  estimation  versus  sample  size  as  a  function  of  the  method 
of  estimation  requires  further  investigation. 


3.  3.  8  Model  Exercise 

The  system  effectiveness  vector  is  now  calculated  using 
the  model  equations.  Variations  in  accountable  factors  may  be  made  to 
determine  the  effect  on  the  end  result.  System  chauige  analysis  may  also 
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be  accomplished  for  product  improvement  or  optimization,  and  the  influence 
on  system  effectiveness  may  be  estimated.  Completion  of  the  model  and  its 
exercise  forms  the  principal  vehicle  for  effectiveness  decision-making. 
These  decisions  may  vary  from  electing  to  proceed  from  the  Conceptual  to 
the  Program  Definition  Phase  of  a  project,  or  deciding  to  modernize  an 
existing  system  by  replacement  of  a  major  component. 

Section  IV  of  this  volume  presents  a  detailed  treatment  of 
effectiveness  model  development.  An  illustrative  example  is  described 
which  follows  each  of  the  eight  tasks. 

3.  4  Discussion  of  Tasks  by  Program  Phase 

The  objectives  of  effectiveness  evaluation  will  differ  depending  on  the 
program  phase.  In  the  Conceptual  Phase  an  estimation  of  probable  effec¬ 
tiveness  may  be  one  of  the  most  important  factors  influencing  the  decision 
whether  or  not  to  proceed  to  program  definition.  Low  levels  of  expected 
effectiveness  may  indicate  desirability  of  considering  alternate  approaches 
to  meeting  the  requirement.  In  the  Program  Definition  Phase,  effectiveness 
considerations  should  influence  the  choice  of  major  system  elements  and 
their  configuration.  Whether  redundancy  will  be  required,  how  the  major 
elements  of  hardware,  personnel,  procedures,  and  logistics  maybe 
optimized  for  maximum  effectiveness,  or  how  to  meet  minimum  acceptable 
levels  at  least  total  cost  are  questions  that  require  resolution. 

In  the  Acquisition  Phase,  updated  estimates  of  effectiveness  should  be 
made  frequently  to  measure  achievement  of,  or  growth  toward,  the 
specified  figure  of  merit.  Detailed  tradeoffs  within  the  subsystems,  equip¬ 
ments,  and  modules  (parts)  will  be  required  to  meet  the  requirements  or 
to  determine  a  best  compromise  (resource  allocation). 

Effectiveness  evaluation  during  the  Operational  Phase  may  form  the 
basis  for  design  changes,  a  product  improvemen<  program,  or  modification 
in  the  support/logistics  struct'ire.  Comparisons  with  other  existing  or 
proposed  systems  may  be  made,  and  decisions  as  to  force  structure,  deploy¬ 
ment,  etc.  ,  may  be  required  to  optimize  mission  objectives.  Regardless  of 
the  objective  of  the  evaluation,  a  logical  procedure  should  be  followed  during 


16 


the  process.  Certainly  the  amount  of  detail  will  differ  greatly  depending 
on  the  objective  and  available  information,  but  the  formalized  steps  should 
nevertheless  be  followed  In  actual  practice  information  obtained  during  one 
phase  may  remain  constant  or  may  be  augmented  and  refined  during  subse¬ 
quent  phases. 

The  following  paragraphs  provide  some  additional  discussion  of  the 
eight  tasks  listed  above  ar,  they  may  differ  during  the  four  major  system 
phases. 

3.  4.  1  Mission  Definition 

Generally  the  mission  definition  originates  from  a  need  or 
requirement  that  forms  the  basis  for  entering  the  Conceptual  Phase.  By 
the  end  of  the  Conceptual  Phase  the  Specific  Operational  Requirements  form 
the  basis  for  mission  definition.  Thi.s  definition  may  be  clarified  during 
the  Program  Definition  Phase  and  more  detail  relative  to  alternatives  may 
become  known.  The  defined  mission  changes  little  during  the  subsequent 
phases  except  as  external  factors  or  constraints  arise,  such  as  a  major 
state-of-the-art  breakthrough,  allowing  an  extension  of  the  mission 
objectives,  or  a  new  enemy  capability  that  may  require  modifications  to 
the  original  mission. 

3.  4.  2  System  Description 

During  the  Conceptual  Phase  very  little  detail  will  normally  be 
available  describing  hardware  and  software  elements  of  the  system. 
Information  will  probably  include  a  block  diagram  identifying  major  system 
elements,  such  as  tracking  radar,  receiver,  display  unit,  etc.  .  During  pro¬ 
gram  definition,  the  major  hardware  end  items  will  be  defined,  modes  of 
operation  determined  and  a  time  line  of  the  mission  profile  developed. 
Physical  and  environmental  factors  can  be  estimated  and  some  knowledge 
of  the  maintenance  support  and  logistics  plan  will  be  available.  During  the 
Acquisition  Phase,  complete  detail  of  tlie  hardware  system  down  to  piece 
part  identification  will  become  available.  The  detailed  system  operating 
plan  and  maintenance  task  analysis  will  have  been  developed.  By  the  early 
Operational  Phase,  complete  descriptive  information  on  all  hardware, 
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procedures,  operation  and  maintenance  schedules,  and  logistics  plans 
should  be  available.  Effectiveness  evaluation  will  require  continued  up¬ 
dating  of  this  information  as  continued  changes  are  proposed  and  adopted. 
Frequently,  each  new  system  placed  in  operation  will  have  a  unique  configu¬ 
ration. 

For  an  illustration  of  the  successive  detail  required  in  system 
description,  see  Example  B,  "Intercontinental  Ballistic  Missile  Squadron,  " 
in  Volume  III. 

3.  4.  3  Specification  of  Figure(s)  of  Merit  (FOM) 

The  selection  of  a  figure  of  merit  will  largely  depend  upon  the 
particular  system  under  evaluation.  For  a  radar  this  may  be  the  proba¬ 
bility  of  target  detection  or  the  probability  of  successful  (accurate)  track. 
For  an  ICBM  fleet  this  figure  may  be  the  expected  number  of  targets  des¬ 
troyed.  Other  examples  of  appropriate  figures  of  merit  are  described  in 
Section  V,  "An  Illustrative  Example,  "  and  in  the  Technical  Supplement, 
Volume  III. 

At  the  Conceptual  Phase,  the  FOM  chosen  will  be  general.  It 
may  consist  of  a  qualitative  statement  as  to  the  probability  of  accomplishing 
the  system  requirement  or  mission. 

During  the  Program  Definition  Phase,  the  FOM's  and  their 
prime  factors  and' subfactors  will  be  identified  with  more  complete  state¬ 
ments  relative  to  the  units  of  measurement  and  appropriate  conditions  under 
which  the  evaluation  is  made.  As  the  system  description  becomes  more 
detailed  during  the  Acquisition  Phase,  FOM's  and  their  prime  A,  D,  C 
(availability,  dependability,  and  capability)  factors  must  be  defined  at 
various  levels  ranging  from  system  to  subsystem  to  equipment  on  down  to 
the  module  and  piece  part  level. 

During  the  Operation  Phase  little  change  will  normally  be  made 
to  the  finally  selected  and  refined  figures  of  merit. 

3.  4.  4  Identification  of  Accountable  Factors 

This  task  involves  clear  recognition  of  the  data  constraints  that 
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may  be  imposed  due  to  the  phase  of  the  program,  the  cost  of  acquiring  data, 
and  the  level  of  detail  that  is  required  in  the  effectiveness  evaluation. 
Sources  of  required  data  must  also  be  identified.  For  example,  will  the 
analysis  require  gross  system  information,  or  will  detail  down  to  the  piece 
part  level  be  needed?  Where  will  this  data  come  from? 

In  the  Conceptual  Phase,  very  little  information  on  personnel, 
procedures,  hardware,  and  logistics  will  normally  be  available.  Again, 
heavy  dependence  will  be  placed  on  estimates  and  extrapolations  from  other 
related  programs.  The  accountability  will  generally  be  to  the  system 
functional  block  diagram  level.  Crude  estimates  of  personnel  requirements, 
the  maintenance  concept,  logistic  considerations,  etcr,  can  be  made. 

During  the  Program  Definition  Phase,  accountability  will 
normally  extend  to  the  principal  end-items.  Information  needed  will 
include: 

(1)  the  environmental  conditions  surrounding  both  the  use  of, 
and  data  collection  on,  these  items.  {Reference  7) 

(2)  generic  reliability  and  maintainability  data. 

(3)  failure  rate  estimates  on  parts  for  which  generic  data 
does  not  exist. 

(4)  detailed  support  policy. 

(5)  numbers  and  skill  levels  of  personnel. 

(6)  maintenance  facilities. 

(7)  maintenance  equipment  and  tools. 

(8)  critical  logistic  support  considerations. 

(9)  system  monitoring  and  exercising  requirements. 

During  the  Acquisition  Phase,  additional  factors  and  refinement 
of  previous  information  should  be  obtained,  such  as: 

(1)  generic  failure  data  for  specific  parts  in  specific 
applications. 

(2)  failure  experience  during  developmental  testing. 

(3)  operating  time  during  design  and  development. 

(4)  modifications  to  the  mission. 

(5)  refined  support  information. 
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(6)  clarification  of  operational  environment. 

In  the  Operational  Phase,  it  is  generally  possible  to  obtain  com¬ 
plete  information  on  all  accountable  factors.  However,  the  method  of 
obtaining  the  information  may  be  costly,  may  interfere  with  system  opera¬ 
tion,  or  may  not  be  relevant  to  the  available  decisions  and  degrees  of 
freedom  of  action.  Seldom  at  the  Operational  Phase  will  a  complete  detailed 
analysis  of  effectiveness  be  made  down  to  the  piece  part  level,  except  where 
a  major  contribution  to  final  system  effectiveness  is  isolated  or  suspected. 

3.  4.  5  Model  Construction 

Section  IV  of  this  volume  traces  the  steps  of  model  construction 

in  detail. 

The  level  of  the  detail  in  model  construction  is  directly  related 
to  the  system  phase.  During  the  Conceptual  Phase,  the  model  may  be  simple. 

During  the  Program  Definition  Phase,  the  detail  should  extend  to 
the  principal  system  hardware  elements  with  submodels  at  the  subsystem 
level  and  for  each  of  the  principal  effectiveness  parameters. 

During  development  and  early  production  (Acquisition  Phase), 
the  model  will  normally  contain  the  most  detail,  since  alternate  decisions 
and  tradeoffs,  design  changes,  and  continuing  configuration  updating  will 
involve  great  detail  in  the  relevant  information  from  which  the  model  is 
constructed. 

During  the  Operational  Phase,  model  inputs  may  be  limited  to 
the  information  from  Air  Force  reporting  systems,  such  as  the  AFM  66-1 
Maintenance  Data  Collection  System,  AFM  65-110  Aircraft  Status  Reporting, 
and  like  data  sources. 

3.  4.  6  Data  Acquisition  and  Parameter  Estimation 

For  this  task,  the  largest  difference  in  detail  among  the  four 
system  phases  is  apparent.  Data  requirements,  test  and  observation  methods 
to  be  employed,  and  data  collection  and  processing  systems  are  obviously 
tailored  to  the  particular  situation  and  the  inevitable  constraints.  In  the 
Conceptual  Phase,  this  information  may  come  entirely  from  previous 
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systems,  be  derived  from  generic  sources,  or  developed  from  basic 
physical  laws. 

During  the  Program  Definition  Phase,  parameters  are  generally 
predicted  using  generic  data  sources,  or  historic  records  of  similar  hard¬ 
ware  items. 

During  development  and  production,  this  information  is  refined 
and  supplemented  by  proofing  and  category  tests  at  the  system  and  equip¬ 
ment  level  and  by  more  extensive  in-process  tests  of  assemblies  and  parts 
by  their  respective  manufacturers.  Assumptions  or  other  provisions  must 
be  stated  that  relate  these  parameter  estimates  to  the  probable  end-use 
environment. 

During  the  Operational  Phase,  parameter  estimation  will  be 
made  from  actual  operating,  failure,  and  maintenance  data.  However,  the 
usefulness  of  present  information  for  this  purpose  is  questionable.  Care¬ 
fully  directed  observation  programs  are  needed  that  are  tied  closely  to  the 
model  requirements  so  that  the  data  derived  may  be  effectively  translated 
into  the  required  parameters. 

3.  4.  7  Model  Exercise 

The  extent  of  model  exercise  and  the  amount  of  information 
derived  is  of  course  dependent  upon  the  previous  tasks.  Results  of  the 
model  exercise  by  phases  may  vary  from  a  single  point  estimation  of 
effectiveness  with  low  confidence  (Conceptual  Phase)  to  an  elaborate  rea  ut 
of  information  on  systems,  subsystem,  equipment,  etc.,  including  estimates 
of  effectiveness  factors  and  their  elements  as  well  as  parameter  variation 
analysis  and  system  change  analysis  (during  the  later  portion  of  the 
Acquisition  Phase). 


SECTION  IV 


MATHEMATICAL  MODEL  CONSTRUCTION 

4. 1  Mathematical  Expression  for  Effectiveness 

In  Section  3.  3.  3,  System  Effectiveness  E  was  defined  as  the  vector 
of  figures  of  merit  specified  for  a  given  system.  In  Section  3.  3.  5,  a 
structure  for  evaluating  this  vector  was  developed.  This  structure  is 
based  on  an  enumeration  of  a  number  N  of  significantly  different  system 
states,  L  .  .  .  ,  n.  The  structure  is  composed  of  three  distinct  parts,  an 
availability  vector  A,  a  dependability  matrix  [D],  and  a  capability  matrix 
[C].  The  elements  of  E,  A,  D,  and  C  are  defined  respectively  as  follows: 

th 

e^  is  the  value  of  the  k  figure  of  merit 

a.  is  the  probability  that  the  system  is  in  state  i  at  the 
beginning  of  the  mission 

d..  is  the  probability  that  the  effective  state  of  the  system 
during  thie  mission  is  j,  given  that  the  mission  was  begun 
in  state  i 

th 

c.,  is  the  value  of  the  k  figure  of  merit,  conditional  on 
^  effective  system  state  j. 

Based  on  these  definitions,  we  may  write 

E  =  A  Cd][C] 

f 

where  A  is  the  transpose  of  A, 
or 

n  n 

®k  .Z  ^i^ij^ik’ 

1=1  j=l  J  - 
4.  2  Availability  Vector 

The  availability  vector  A  is  a  row  vector  [a^^, . a^]  containing  the 

probabilities  of  the  various  system  states  at  that  point  in  time  when  a 
mission  begins. 
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Ws  may  illustrate  the  principles  of  availability  analysis  by  considering 
a  system  for  which  the  following  assumptions  are  a  satisfactory  represen¬ 
tation  of  reality: 


(1)  The  system  is  a  single  "black  box"  which  fails  at 
random  times  following  repair; 

(2)  When  the  system  fails,  it  is  immediately  evident 
and  repair  is  initiated  at  once.  Repair  completely 
renews  the  system; 

(3)  The  system  is  in  operation  when  it  is  not  in  repair; 
and, 

(4)  The  demand  for  system  use  occurs  sufficiently  long 
after  initial  system  installation  so  that  it  has  undergone 
several  failure -repair  cycles. 


For  this  system  there  are  two  possible  states:  (1)  "operative"  or  (2) 
"failed"  (in  repair). 

I 

The  availability  vector  X  then  consists  of  two  components,  a^^  and  a^: 

5  =  [aj  aj 


where 


=  probability  that  system  is  operable  at  a  random  point 
in  time 

a^  =  probability  that  system  is  in  repair  at  a  random  point 
in  time. 

The  probability  that  it  is  operating  (i.  e.  ,  in  State  1) 
will  be  given  by 

Availability  =  a, 

(Eq.  1) 

_ (mean  time  to  failure) _ 

(mean  time  to  failure)  +  (mean  time  to  repair) 

and  the  probability  that  it  is  in  repair  (i.  e.  ,  in  State  2)  will  be  given  by 

Unavailability  =  a_ 

(Eq.  2) 


(mean  time  to  repair) 

(mean  time  to  failure)  +  (mean  time  to  repair) 
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That  is,  our  best  guess  for  is  the  expected  fraction  of  time  that  the 
system  will  be  non-failed  and  our  best  guess  for  a^  is  the  expected  fraction 
of  the  time  that  the  system  will  be  in  repair.  Therefore,  if  we  have  the 
chronological  past  history  of  the  system  we  can  estimate  a,  and  a,  by 

JL  u 

A  _  (total  time  operating) 

1  (total  time  operating)  +  (total  time  in  repair) 

A  _ _ (total  time  in  repair) _ 

**2  (total  time  operating)  4-  (total  time  in  repair) 

However,  not  all  systems  are  observed  continuously.  For  example,  consi-  , 
der  a  single  unit  that  is  checked  out  by  a  series  of  tests  and  then  placed  in 
an  active,  but  unattended  and  unmonitored  state  for  a  constant  period  of 
time  T.  Assume  that  this  cycle  of  checkout  and  standby  repeats  itself  in¬ 
definitely.  It  may  fail  during  the  time  T,  and  if  it  does,  the  failure  will  not 
be  discovered  until  the  time  T  has  elapsed  and  a  new  series  of  checkouts  is 
undertaken.  We  say  that  such  a  system  is  periodically  checked,  and  wf 
may  express  the  availability  of  this  system  by, 

a^  =  Availability 

(expected  time  non-failed  in  T) 

~  (duration  of  '?)  +  (time  down  in  checkout/repair) 

This  expression  assumes  that  repair,  if  it  occurs,  extends  the  time  down 
beyond  that  required  to  check  out  a  non-failed  system  and  that  the  expected 
time  non-failed  in  T  is  not  dependent  upon  the  age  of  the  system,  although 
system  aging  (wear out)  can  be  included  by  making  a  somewhat  more  com¬ 
plicated  calculation. 

Practical  systems  are  usually  composed  of  a  number  of  single  units  or 
black  boxes.  Consider,  for  example,  a  system  for  which  the  following 
assumptions  constitute  a  satisfactory  approximation  to  reality  during  pre¬ 
mission  life: 

(1)  There  are  N  units  which  fail  independently  with  mean 

times  to  failure  t^  ,  . .  .  ,  t^  that  do  not  change  with  system 
age.  1  N 
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(2)  When  any  one  of  these  units  fails,  it  is  immediately 
evident  and  repair  is  initiated  at  once  (continuously 
monitored  system  with  unlimited  repair  facilities). 

Repair  times  have  stationary  distributions  with 
means  t  ,  . . .  ,  t 

"^1 

(3)  Any  units  which  are  not  undergoing  repair  are  in 
use. 

(4)  The  mission  is  started  at  a  random  point  in  time 
sufficiently  long  after  initial  system  operation  so 
that  each  unit  has  undergone  several  failure -repair 
cycles. 

For  this  system,  each  unit  has  two  possible  states,  operative  (O)  or 
undergoing  repair  (1).  There  are  therefore,  at  most  2^  system  states,  each 
of  which  may  be  represented  by  n  binary  digits  corresponding  to  the  compo¬ 
nent  states,  although  a  judicious  lumping  together  of  similar  system  states 
will  reduce  the  total  number  of  states  v'hich  must  be  explicitly  represented 
in  the  analysis. 


The  availability  of  any  one  unit  of  the  system  is  given  by  this 
modification  of  Equation  1: 


(Eq.  3) 


th 

a.  =  (mean  time  to  failure  of  i  unit) 
^  .tH 


:tF 


(mean  time  to  failure  of  i  unit)  +  (mean  time  to  repair  i  unit) 


'  Je  might  define  a^^  as  the  probability  that  all  N  units  are  available.  Thus, 
a^^  would  be  calculated  from  Equation  3  by: 


N 

a,  =  n  O'. 

1  i=i  * 


The  second  state  of  the  system  might  be  defined  as  the  probability 
that  all  units  but  the  first  are  available.  This  would  be  calculated  from 
Equation  3  by: 

N 

=  (1  -  o-i)  IT 
1=2 

This  process  of  combining  the  ^i  would  be  continued  until  all  significant 
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system  states  had  been  accounted  for. 


Models  for  computing  the  elements  of  the  availability  vector  must 
take  into  account  failure  and  repair  time  distributions,  preventive 
maintenance  and  miscellaneous  down-time  schedules,  checkout  procedures, 
personnel  deployment,  spare  parts,  supply  facilities  as  well  as  transporta¬ 
tion  and  various  administrative  actions.  (References  8,  9,  10) 


The  following  list  of  possible  system  states  should  be  referred  to  when 
establishing  availability  models; 

(1)  Assigned  to  alert/standby  and  non-failed 

(2)  A-ssigned  to  alert/standby  and  failed  in  a  manner 
detectable  by  field  test 

(3)  Assigned  to  alert/ standby  and  failed  in  a  manner 
undetectable  by  field  test 

(4)  On  alert/standby  and  waiting  for  checkout/ 
diagnosis 

(5)  Off  alert/standby  and  in  checkout/diagnosis 

(6)  Off  alert/standby  and  waiting  for  spares. 

4.  3  Dependability  Matrix 

The  availability  vector  presents  a  picture  of  the  condition  in  which  the 
system  is  likely  to  be  found  at  the  beginning  of  a  mission.  The  next  step  in 
analysis  requires  a  representation  of  the  system  during  the  course  of  a 
mission,  conditional  on  its  state  of  readiness  at  the  beginning. 


The  dependability  matrix  is  a  square  array  of  numbers.  In  general, 
if  there  are  n  significant  system  states,  the  dependability  matrix  has  n  rows 
and  n  columns 


[D]--: 


‘^ll  ‘^iz  • 
^21  ^22  ‘ 


d  ,  d  .  d 
nl  n^  nn 
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■is. 


I 


where 


i 


I  d..  =  l, 

J=i  *' 


If  no  repair  is  possible  during  the  mission  some  of  the  d's  will  be  zero.  If 
the  states  are  numbered  in  order  of  increasing  degradation,  the  matrix  is 
triangular 


[D]  = 


^1^2- 


0  d 


22* 


O  O  .  .  .  d 

nn 


The  specific  formulation  of  the  dependability  matrix  will  depend  upon 
the  effect  of  failures  during  the  mission  and  whether  or  not  repair  is 
possible  during  the  mission.  For  example,  consider  three  different  single 
unit  systems  which  carry  out  missions  of  duration  T,  each  having  the 
following  characteristics: 

System  a  (for  example,  an  ICBM) 

(1)  No  repair  possible  during  mission 

(2)  System  state  at  end  of  mission  is  only 
important  mission  criterion 

System  b  (for  example,  a  G.  C.  A.  system) 

(1)  Repair  is  possible  during  mission 

( 2)  System  state  at  end  of  return  of  air  - 
craft  from  mission  is  only  irhportant 
mission  criterion 


System  c  (for  example,  doppler  navigation  radar) 

(1)  Limited  repair  is  possible  during  mission 

(2)  Fraction  of  time  out  of  commission  during 
mission  is  mission  criterion  (e.  g.  ,  posi¬ 
tional  accuracy  degradation). 
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In  these  examples  assume  that  the  system  must  be  in  either  one  of 
two  states  at  the  time  of  demand  (mission  initiation):  namely,  operable  (1) 
or  failed  (2).  Thus,  we  consider  a  2  x  2  dependability  matrix: 


We  place  the  following  interpretations  on  the  d's  for  systems  a  and  b: 

dj^j^  =  The  probability  that  the  system  is  operable  at  the  end 
of  the  mission,  given  that  it  was  operable  at  the  start 
of  the  mission 

dj^2  =  The  probability  that  the  system  is  failed  at  the  end  of 
the  mission,  given  that  it  was  operable  at  the  start  of 
the  mission 

d^j^  =  The  pro^‘ ability  that  the  system  is  operable  at  the  end 
of  the  mission,  given  that  it  was  failed  at  the  start  of 
the  mission 

d22  =  The  probability  that  the  system  is  failed  at  the  end  of 
the  mission,  given  that  it  was  failed  at  the  start  of 
the  mission. 

Now,  since  there  can  be  no  repair  for  system  a  during  the  mission, 
dj^j^  is  simply  the  conventional  reliability  of  the  system,  so  that  if  failures 
are  random  and  mean  time  to  failure  does  not  depend  upon  system  age, 
the  following  relationship  holds 


XT 


e 


X  =  system  failure  rate 
T=  duration  of  mission 


and 


XT 


Also,  since  repair  is  not  possible  during  the  mission,  a  system 
failed  at  the  initiation  of  the  mission  must  still  be  failed  at  the  end  of  the 
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missJon;  therefore: 


1 


and 


=  O 


Hence,  the  dependability  matrix  for  system  a  is 

= 

Although  we  interpret  the  d's  in  the  same  manner  for  system  a  and  b, 
their  calculation  is  not  the  same,  since  we  allow  for  the  possibility  of 
repair  during  the  mission  for  system^b. 

For  simplicity,  let  us  assume  that  times  to  failure  after  repair 
actions,  and  times  to  repair  after  failures  are  exponentially  distributed  for 
system  b.  This  means  that  the  probability  of  a  failure  or  a  repair  in  a  small 
increment  of  tim;  At  can  be  expressed  as 

probability  of  failure  in  time  At  =  XAt 

probability  of  a  repair  in  time  At  =  pAt 

where 


-XT 


O 


1  -  e 


-XT 


X  =  system  failure  rate 
M-  =  system  repair  rate. 


Then  we  may  write: 


IL 


t  +  At 


=  P, 


(1  -  XAt)  +  P, 


2L''J 


pAt. 


This  statement  reads:  j^Probability  that  the  system  is  operable  at  time 
t  +  At  ,  =  Improbability  that  the  system  was  operable  at  time  t  j,  x  j^Proba- 
bility  that  the  system  does  not  fail  in  the  time  increment  At^j  +  Probability 
that  the  system  was  failed  at  time  tj  x  |^Probability  that  a  repair  is  completed 
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1  r 

in  the  time  increment  AtJ.  Subtracting  Pj  t  J  from  both  sides  of  this 
equation,  dividing  both  sides  by  At,  and  taking  the  limit  as  At-K),  we  have 
from  the  fundamental  definition  of  a  derivative,  that 

lim  pJ  t  +  At  -  PJ  t  I  d  P,  t 
At  -»  O 


But  sinc'  the  system  can  only  be  either  failed  or  non-failed, 


(Eq.  4) 


(Eq.  5) 


Combining  Equations  4  and  5 


r  • 

^  =-{X+  M.)  Pi[tJ+  IX 


The  general  solution  to  this  differential  equation  is  found  to  be 


.[tl-  . 

x+ir  L  J 


+  Pl[oJe 


1-{^  +  |i)t 


(Eq.  6) 


where  is  the  probability  that  the  system  is  operable  at  time  t  =  O. 

If  we  identify  t  =  O  as  the  time  of  initiation  of  the  mission, then  d^^^^  is  the 
value  of  P^tJ  when  Pj^j^o]  is  set  equal  to  1,  and  d22^  is  the  value  of  P^I^T  j 
when  P^  Oj  is  set  equal  to  zero,  where  T  is  the  duration  of  the  mission. 


Thus. 


Jb). _ L  ,  ^  |X)T 

^11  A.  +  p,  ^  X  +  p 


12  "  r+  p 


(b)  _  IX 

21  "  TTT 


■{X+  p)T 


l.,-(X+^)T 
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M- 

XTII 


,{b)  . 

h'L  - 


\ 

A.  +  H 


^i)T 


For  system  c,  we  place  the  following  interpretations  on  the  d's  of  the 
matrix: 

=  The  expected  fraction  of  time  during  the  mission 
during  which  the  system  is  operable,  given  that 
it  was  operable  at  the  start  of  the  mission 

dj^2  =  "Tbe  expected  fraction  of  time  during  the  mission 
during  which  the  system  is  failed,  given  that  it 
v/a?  operable  at  the  start  of  the  mission 

d^j^  =  The  ■'7  pected  fraction  of  time  during  the  mission 
during  which  the  system  is  operable,  given  that 
it  was  failed  at  the  start  of  the  mission 

d22  =  The  expected  fraction  of  time  during  the  mission 

during  which  the  system  is  failed,  given  that  it 
was  failed  at  the  start  of  the  mission. 


We  may  calculate  these  probabilities  directly  from  Equation  6.  If 
Pfl^tJ  is  the  probability  that  the  system  is  in  state  i  at  time  t,  then  the 
expected  time  t  spent  in  state  i  in  an  interval  of  length  T  is  given  by: 


t  = 


.  T 
O 


dt 


Thus,  the  expected  fraction  of  time  spent  in  that  state  in  an  interval  of 
length  T  is  given  by: 


1  =  1 1^  pit  1 
T  T  Jq  ^iLM 


dt 


Thus  dj^j^  for  system  c  is  obtained  from  d^^  of  system  b  by 


11 


(c)  .T  (b) 

^11  ""  T  Jq  ^11 


M*  ,  X 

-f-  X 

X+  |i  (>.+  IJ.)  T 
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and  similarly: 


=  1  -  d 


11 


(c) 


X 

X  +  |ji 


X 

{X+ 


1  _  e-{^+  ^i)T 


-JL_ 

X  +  |J, 


1 

tt+htt- 


l_e-(^+^)T 


=  1  -  d 
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(c) 


X  ,  [L 

- +  - ^ 

X  +  (X  +  P-)  T 


4.  4  Capability  Matrix 

th 

The  element  of  a  capability  matrix  is  the  k  figure  of  merit 
associated  with  system  performance  in  effective  system  state  j. 

The  magnitude  and  dimensions  attached  to  this  figure  of  merit  depend 

upon  the  specific  nature  of  the  system  undergoing  evaluation.  For  example, 

system  a  above  could  be  an  ICBM,  Then  c.  might  be  the  expected  number 

J 

of  targets  destroyed,  given  that  j_ICBM's  are  delivered  to  the  target  area. 
Calculation  of  each  c^  could,  in  this  case,  require  an  accounting  for  the 
targeting  policy,  guidance  dispersion,  warhead  yield  pattern,  target  area, 
target  hardness,  propellant  depletion  probability,  enemy  countermeasures, 
and  penetration  aids. 

System  b  could  be  a  Ground  Controlled  Approach  (GCA)  radar  landing 

system.  This  system  is  only  needed  in  adverse  weather  for  aircraft 

returning  to  base.  Under  conditions  of  very  low  ceiling  and  visibility  over 

an  extensive  geographical  area,  with  the  system  inoperative  there  is  a  high 

probability  that  returning  aircraft  would  be  unable  to  land  safely,  and  hence, 

v/ould  be  lost.  Thus,  in  this  case,  we  might  define  our  c.  as  follows: 

J 

Cj^  =  The  probability  that  a  returning  aircraft  would 

land  safely,  given  that  the  GCA  system  is  operating 

c^  =  The  probability  that  a  returning  aircraft  would  land 
safely,  given  that  the  GCA  system  is  inoperable. 
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Thus,  in  this  case,  the  c.  are  probabilities.  Notice  that  the  capa- 

J 

bility  vector  here  is  a  measure  of  the  degree  of  compatibility  between  two 
major  systems,  one  of  which  is  airborne,  and  the  other  of  which  is  a 
ground  support  system. 


The  prime  objective  of  the  airborne  system  is  to  reach  the  target, 
destroy  it,  and  return  to  the  vicinity  of  the  base;  the  prime  objective  of  the 
ground  system  is  to  bring  the  aircraft  in  safely.  Clearly,  the  methods  pre¬ 
sented  here  are  not  restricted  to  consideration  of  a  single  system. 


Because  the  c.  depend  so  specifically  on  the  type  of  analysis  being 
J 

performed,  we  shall  defer  examples  of  specific  capability  matrix  develop¬ 
ment  to  the  example  which  follows  this  discussion. 


By  noting  the  performance  of  this  synthetic  population,  predictions 
(assessments)  can  be  made  of  system  effectiveness  for  the  anticipated 
population. 


4.  5  Simulation 


In  highly  complex  systems,  realistic  assumptions  relating  to  the 
accountable  factors  often  make  the  analytical  formulation  of  Availability  (A), 
Dependability  (D),  and  Capability  (C)  matrices  impractical.  When  this  is 
the  case,  the  only  feasible  course  is  to  resort  to  simulation  techniques 
using  either  analog  or  digital  computers,  or  both.  (References  11,  12,  13, 

14) 

Simulation  methods  available  are  so  nui  rous  and  varied  that  it  is 
impossible,  here,  to  give  a  preferred  method.  The  best  method  in  a  par¬ 
ticular  case  depends  on  the  nature  of  the  system,  the  phase  of  the  program, 
and  the  precision  required.  For  example,  in  some  cases  it  may  be  desir¬ 
able  to  use  simulation  methods  only  to  provide  estimates  of  the  A,  D,  C 
matrix  elements;  in  other  cases  it  may  be  preferable  to  by-pass  the 
intermediate  outputs  and  proceed  directly  to  an  overall  measure  of 
effectiveness. 

Despite  the  possible  variations,  all  simulation  methods  for  estimating 
effectiveness  have  some  fundamental  common  characteristics.  First,  the 
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relations  between  accountable  factors  and  the  effectiveness  figure  of  merit 
must  be  mathematically  described  (References  15,  16):  second,  the  manner 
in  which  the  accountable  factors  may  vary  from  one  system  trial  to  another 
must  be  known  or  reasonable  assumptions  established;  and  third,  a  large 
number  of  repeated  system  trials  must  be  run  on  the  computer  using  ran¬ 
domly  selected  values  of  the  accountable  factors,  counting  the  resulting 
system  successes  and  failures.  This  last  step  is  commonly  referred  to  as 
a  Monte  Carlo  procedure. 

Simulation  techniques,  like  analytical  methods,  can  be  used  to  deter¬ 
mine  the  sensitivity  of  the  system  figure  of  merit  to  variations  in  the 
accountable  factors.  In  such  an  exercise  deliberate  (rather  than  random) 
variations  are  introduced  in  the  expected  values  of  the  accountable  factors 
and  the  Monte  Carlo  process  is  repeated. 

In  Section  5.  3  the.  use  of  simiilation  techniques  is  illustrated 
through  application  to  a  specific  effectiveness  problem. 


SECTION  V 


AN  ILLUSTRATIVE  EXAMPLE 


5.  Introduction 

In  the  foregoing  sections  the  general  framework  of  system  effectiveness 
has  been  established  and  the  tasks  necessary  to  effectiveness  prediction  and 
analysis  have  been  delineated.  In  this  section  the  general  techniques  de¬ 
scribed  in  Section  III  will  be  illustrated  in  detail  through  application  to  a 
specific  example.  The  example  is  approached  first  in  a  fundamental 
analytical  way.  This  is  followed  by  an  illustration  of  the  use  of  Monte 
Carlo  simulation  in  the  prediction  and  analysis. 

5.  1  Analytical  Method 

The  primary  value  of  an  analytical  approach  is  that  it  provides  an 
insight  between  system  parameters  and  their  relative  impact  on  system 
effectiveness.  Before  discussing  the  details  of  the  example,  certain  general 
comments  and  precautions  are  in  order: 

(1)  In  selecting  appropriate  figures  of  merit,  their  fundamental 
purpose  must  be  kept  clearly  in  mind  --to  provide  manage¬ 
ment  with  information  on  which  to  base  decisions.  The 
numerical  value  of  the  figure  of  merit,  when  compared  to 
quantitative  system  requirements,  indicates  whether  or  not 
corrective  action  is  necessary.  The  sensitivity  of  the 
figure  of  merit  to  combinations  of  variations  in  system 
parameters  provides  a  technical  basis  for  taking  corrective 
action. 

(2)  A  prediction  is  no  better  than  the  data  on  which  it  is  based 
and  the  quality  of  the  data  is  not  just  a  matter  of  numerical 
accuracy.  Specifically,  data  for  a  particular  application 
must  reflect  a  similarity  between  conditions  under  which  the 
data  were  taken  and  expected  operating  conditions  of  the 
system  under  evaluation.  All  prediction  processes  require 
the  interpolation  or  extrapolation  of  data,  but  the  smaller 
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the  degree  of  extrapolation  the  better  the  prediction  will  be. 
For  example,  the  data  sources  selected  for  this  illustration 
are  derived  from  the  study  of  ground  based  electronic 
equipment  quite  similar  to  the  hypothetical  system  under 
consideration.  The  absence  of  comprehensive  generic  data 
sources  is  currently  the  weakest  link  in  the  chain  of 
effectiveness  prediction. 

Proceeding  with  the  details  of  the  example,  a  very  simple  ground 
based  radar  system  has  been  hypothesized  for  the  illustration.  (See  Figure  3). 


FIGURE  3 

SCHEMATIC  OF  GROUND  BASED  RADAR  SYSTEM 

5. 1. 1  Mission  Definition 

The  system  should  detect  target  aircraft  above  the 
horizon  line  of  sight  at  ranges  up  to  200  miles  and  while  the  target  is  within 
this  maximum  range,  track  it  in  range  and  azimuth  within  admis sable  error 
presenting  this  information  to  an  operator  at  the  site, 

5.1.2  System  Description 

The  hypothetical  system  shown  in  Figure  3  is  used  for 
this  example.  It  consists  of  two  transmitters  in  parallel,  an  antenna,  a 
receiver,  a  display  and  synchronizer,  and  an  operator. 


36 


5. 1.  3  Specification  of  Figures  of  Merit 


To  show  the  flexibility  of  the  basic  mathematical 
frame  work,  four  figures  of  merit  are  chosen: 

(1)  Probability  of  target  detection  at  maximum 
range  at  a  random  point  in  time. 

(2)  Probability  of  detection  at  maximum  range 
and  continuous  tracking  within  required 
accuracy  during  a  prescribed  time  period 
(assumed  to  be  30  minutes  in  this  example). 

(3)  Probability  of  continuous  tracking  within 
required  accuracy  during  prescribed  time 
period;  given  successful  detection. 

(4)  Curve  of  probability  to  detect  and  track 
versus  range  (of  special  interest  if  the 
probability  of  detection  at  maximum  range 
is  less  than  the  goal). 

5. 1.  4  Identification  of  Accountable  Factors 

The  factors  to  be  explicitly  included  in  the  computation 
of  the  figures  of  merit  are: 

(1)  Subsystem  failure  rates 

(2)  System  and  subsystem  repair  times 

(3)  Application  of  maintenance  manpower 
(unlimited  resources  assumed). 

(4)  Repair  supplies  availability  (unlimited 
resources  assumed). 

(5)  System  configuration  (redundancy  and  pro¬ 
vision  for  emergency  modes  of  operation). 

(6)  Maximum  range  for  successful  target 
detection. 

(7)  Radar  transmitter  power. 

5.1.5  Model  Construction 

The  first  step  in  the  analytical  structuring  of  the  model  is 
the  definition  of  significant  states  of  the  system,  based  upon  possible  mission 
outcomes.  This  step  is  followed  by  construction  of  the  availability,  depend¬ 
ability,  and  capability  matrices,  and  finally  the  matrices  are  multiplied 
appropriately  to  give  the  required  figures  of  merit. 


37 


5. 1 . 5. 1  Definition  of  System  States 


State 

Designator 

1 

2 


5.  1.5.2 


Definition 

All  units  operating  properly 

One  transmitter  inoperative,  the  other 
transmitter  and  all  other  units  operat¬ 
ing  properly. 

System  totally  inoperative  due  to 
inoperative  condition  of  both  trans¬ 
mitters  or  any  one  of  the  other  units. 

Constructing  the  Availability  Vector 


As  defined  in  Section  4.2,  the  steady  state 
availability  vector  (a  single  row,  three  column  matrix  in  this  case)  is  the 
set  of  probabilities  that  the  system  is  in  state  1,  2,  or  3  at  a  random  point 
in  time;  i.e.,  at  some  unpredictable  time  when  a  target  enters  its  maxi¬ 
mum  range  sector.  Symbolically, 


where 


Availability  =  A  = 

a^^  =  probability  that  the  system  is  in  stare  1 
a^  =  probability  that  the  system  is  in  state  2 
=  probability  that  the  system  is  in  state  3 

5. 1.5.  3  Constructing  the  Dependability  Matrix 


As  indicated  in  Section  4.  3  the  dependa¬ 
bility  matrix  is  a  square  array  of  numbers  (3  rows  and  3  columns  in  this 
case).  Since  it  is  assumed  that  no  repair  is  possible  during  the  critical 
mission  time;  i.e.,  while  tracking  a  target,  some  of  the  elements  will  be 
zero.  Symbolically, 
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djj  =  probability  that  the  system  starting  the  mission  in  state  1 
completes  it  in  state  1  (conventional  reliability). 

=  probability  that  the  system  starting  the  mission  in  state  1 
completes  it  in  state  2. 


^13’  ^22’  ^23’ 


d^^  similarly  defined. 


From  these  definitions  and  the  assumption 
of  no  repair  during  the  mission  it  is  clear,  for  example,  that  ^21""  pro¬ 
bability  of  starting  the  mission  in  state  2  and  completing  it  in  state  1  --  is 
zero. 

5.  1.5.  4  Constructing  the  Capability  Vector 

The  capability  of  the  system  in  this  example 
may  be  defined  as  the  probability  that  it  can,  in  a  given  state,  carry  out  its 
design  functions  at  the  required  times.  Since  the  particular  mission  under 
consideration  is  divided  into  two  time  sequential  phases,  two  capability 
vectors  will  be  required.  They  are: 

Cj(0) 

Capability  of  initial  target  _  -  c  (q) 

detection  and  acquisition  "  ^  ~  I  ~  '^2'  ' 

03(0) 

where 

Cj(0)  =  probability  of  detection  at  a  random  time,  t  =  0  when 
the  system  is  known  to  be  in  state  1. 

C2(0)  =  as  above,  but  known  to  be  in  state  2. 

C^iO)  =  as  above,  but  known  to  be  in  state  3. 
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Cj(30) 


Capability  of  track,  given 
detection  and  acquisition 


=  C(t 


30)  = 


c^OO) 


CjOO) 


where 


Cj(30)  =  probability  of  track  through  30  minutes,  given 

detection  at  t  =  0  and  the  system  in  state  1  at  t  =  30. 

0^(30)  =  as  above,  but  system  in  state  2  at  t  =  30. 

C^(30\  =  as  above,  but  system  in  state  3  at  t  =  30. 

5.  1.  6  Data  Acquisition 


For  this  hypothetical  example,  data  used  for  estimating 

such  parameters  as  failure  rate,  repair  rate,  etc.,  are  derived  from  sources 

4/ 

such  as  those  listed  here;— 


(1)  MIL  Handbook  217  --  electronic  part  reliability 
data  from  which  subsystem  failure  characteristics 
can  be  estimated  when  the  design  is  fairly  com¬ 
plete;  e.  g.,  in  the  Acquisition  Phase. 

(2)  System  Reliability  Prediction  by  P'unction  --  data 
and  methods  from  which  subsystem  failure 
characteristics  can  be  estimated  v/hen  the  design 
is  in  its  preliminary  states;  e.  g.,  in  the  Definition 
Phase. 

( 3)  Maintainability  Measurement  and  Prediction 
Methods  for  Air  Force  Ground  Electronic 
Equipment  --  data  and  methods  from  which  system 
and  subsystem  repair  characteristics  can  be  esti¬ 
mated  when  the  design  is  fairly  complete;  e.  g.,  in 
the  Acquisition  and  Operational  Phases. 

5.  1.  7  Parameter  Estimation 


Estimates  of  the  parameters  used  to  derive  the  basic 
availability  and  dependability  factors  of  the  effectiveness  model  were 


4/ 


See  Bibliography  for  other  data  sources 
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arbitrarily  made  using  such  sources  as  are  listed  above  in  context  with  the 
mission  profile,  system  states  and  accountable  factors  listed  in  Paragraphs 
5.  1.  2,  5.  1.  4,  and  5.  1.  5.  As  discussed  in  Section  4.  4,  there  is  no  general 
method  by  which  elements  of  the  capability  vector  can  be  estimated.  Each 
individual  system  requires  its  own  unique  approach  at  each  phase  of  system 
evolution.  The  methods  used  must  sometimes  be  theoretical,  sometimes 
empirical  and  sometimes  a  combination  of  the  two.  The  example  iinder  dis¬ 
cussion  provides  an  excellent  illustration  of  the  manner  in  which  a  com¬ 
bined  empirical-theoretical  approach  can  be  employed. 

5.  1.  8  Model  Exercise 

5.  1.  8.  1  Availability  Estimates 

It  is  assumed  that  all  units  have  independent 
exponentially  distributed  failure  and  repair  times  with  the  following  mean 
values: 


Each  Transmitter  MTBF  (mean-time-between-failure)  =  40  hours 

Each  Transmitter  MTTR  (mean-time-to-repair)  =  4  hours 

Composite  Antenna,  Receiver,  Display  and  Synchronizer  MTBF  =  lOO  hours 
Composite  Antenna,  Receiver,  Display  and  Synchronizer  MTTR  =  1  hour 

Employing  the  methods  of  Section  4.  2,  the 
availability  vector  elements  are  now  calculated  to  be: 


Appendix  I. 


a^  =  0.  818 
a^  =  0.  149 
a^  =  0.  033 

A  =  0.  818  0.  149  0.  033 

The  details  of  the  calculation  are  given  in 
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5 . 1 . 8 . 2  Dependability  Estimate 


Continuing  as  in  Section  4.  3  and  using  the 
same  assumptions  regarding  the  units  used  for  the  availability  computations 
we  get: 


where 


and 


^  ^  ^  A  A?  R 

T""  (MTBF)^  =  40  "" 


R 


=  each  transmitter  failure  rate 

=  composite  receiver,  cintenna,  display, 
synchronizer  failure  rate 


R,j,(30  min)  =  exp  i-  X,pt  =  exp  ;-.025  x0.5  =  0.988 
Rj^(30  min)  =  exp  -  0.010x0.5  =0. 995 


It  is  now  possible  to  compute  the  individual 
dependability  matrix  elements.  Again  the  detailed  calculations  are 
included  in  Appendix  I .  The  results  are: 

'0.971  0.024  0.005 

D(30min)=  0  0.982  0.018 

0  0  1 


5. 1.8.  3  Estimate  of  Capability 

5.  1 . 8 . 3 .  1  The  Elements  of  the  C(0)  V ector 
The  elements  of  the  C(0)  vector 

represent  the  probabilities  that  the  system  in  a  given  state  can  detect  a 
target  at  maximum  range.  We  shall  assume  that  empirical  data  have  been 
taken  at  the  desired  radar  frequency  with  a  known  transmitter  power  and 
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range  and  with  targets  of  typical  cross  sections  and  receivers  of  typical 
noise  figures.  This  data  then  provides  an  estimate  of  the  probability  of 
detection  at  that  known  power  level  and  range. 

In  Paragraph  3  of  Appendix  I,  the 
following  proportion  is  derived  from  theoretical  considerations  based  upon 
noise  theory  and  the  radar  equation: 


where 

Pj  =  the  probability  of  detection  in  case  1 
l^(x)  =  the  natural  logarithm  of  x 

P  =  transmitter  power  in  case  1 

^  1 

r^  =  range  in  case  1 

p^  =  the  probability  of  detection  in  case  2,  and  so  on. 

For  example,  assume  that  the 

combined  power  of  the  two  transmitters  in  this  illustration  yields,  through 
measurement,  a  probability  of  detection  at  the  full  200 -mile  range  of 
Cj(0)  =  0.9.  Then,  the  probability  of  detection  at  half  power  is  readily 
computed  from  the  foregoing  proportion  to  be  0^(0)  =  0.  683.  Further, 
since  either  no  power  is  transmitted  in  state  3,  or  the  receiver  chain  is 
inoperative,  the  probability  of  detection  must  be  zero  in  that  state,  or 
C3(0)=  0. 

Evidently  this  same  relation  can 
be  used  to  determine  capability  as  a  continuous  function  of  range.  Used  in 
this  fashion  under  the  conditions  of  this  example,  the  relation  yields  the 
following  expressions  for  the  capability  elements  (see  Paragraph  3  of 
Appendix  I  for  mathematical  details). 
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C  j{0,  r)  =  1  -  exp 


2. 3  X  (200)^ 

- 5 - 


C^CO,  r)  =  1  -  exp 


2.3  X  (200)^ 


C^{0,  r)  =  0  . 

5. 1.8.  3. 2  The  Elements  of  the  C(30)  Vector 

The  elements  of  the  C{30)  capa¬ 
bility  vector  represent  the  probability  of  tracking,  with  required  accuracy, 
once  the  target  has, been  detected.  In  the  interests  of  simplicity,  we  shall 
assume  that  0^(30)  =  C2(30).  To  facilitate  a  quantitative  solution  to  the 
problem,  we  shall  assume  that  both  values  are  estimated  to  be  0.98 
{Cj{30)  =C2(30)  =  0.98).  Of  course  03(30)  =  0  . 

5.  1. 8. 3.  3  The  Elements  of  the  Capability 
Vector  in  the  I^resence  of 
^nemy  Countermeasures 

From  the  foregoing  discussions  it 

is  clear  that  with  a  little  ingenuity  the  effects  of  enemy  radar  counter¬ 
measures  could  be  incorporated  into  the  model.  For  example,  the  ability 
to  track  through  jamming  signals  may  depend  on  the  two  transmitters  opera¬ 
ting  at  different  frequencies.  For  each  state  of  the  system,  then,  there  is 
a  different  probability  that  the  system  will  track  properly  in  the  possible 
presence  of  jamming.  To  illustrate  cursorily  how  this  problem  might  be 
analyzed,  let  the  event  represent  successful  tracking  in  state  1,  and  the 
event  B  represent  the  presence  of  jamming,  with  B  the  absence  of  jamming. 
Then  the  unconditional  probability  of  successful  tracking  is 


where 


P{Aj)  =P{Aj 


B)P{B)  -I  P{Aj 


B)P{B) 
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P(A^  B)  is  the  probability  of  successful  tracking  in  the 
I  presence  of  jamming,  in  state  1 

P(A^  B)  is  the  probability  of  successful  tracking  in  the 
absence  of  jamming,  in  state  1 

P(B)  is  the  probability  of  jamming 

P(B)  is  the  probability  of  no  jamming. 

We  shall  not  carry  this  part  of 
the  illustration  further.  It  has  been  introduced  only  to  indicate  further 
possiblities  of  the  model. 

5. 1.8.4  Effectiveness  Estimates 

It  is  now  possible  to  combine  the  foregoing 
matrices  to  establish  effectiveness  models  for  the  figures  of  merit  specified 
in  Section  5.1.3. 


NOTE:  In  this  case  the  general  formulation;  i.e.,  A[qC  simplifies  to  the 
above  since  the  initial  portion  of  the  mission  (detection)  is  con¬ 
sidered  to  occur  instantaneously,  and  consequently  the  dependa¬ 
bility  matrix  reduces  to  the  identity  matrix. 


0 

1 

0 


0 

0 

1 


E2  =  Effectiveness  in  detection  and  track 


E^  =  ^|c(0)]  [D(30r]  ^■(30) 


where 


[C(0)]= 


Cj(0)  0 


0  C2(0)  0 


L 


0  0  €3(0) 


and  other  symbols  as  previously  defined. 

Using  the  numerical  values  previously  derived: 


E^  =  0.818  0. 149  0.033] 


0.900  0  0 

0  0.683  0 

0  0  0 


0.971  0.024  0.005 

0  0.982  0.018 
0  0  1 


0.98] 

0.98 

0 


§ 


=  10.736  0.102  0] 


0.975 
0. 962 
0 


E^  =  0.824  . 


E3  =  Effectiveness  in  track,  given  target  detection  and  acquisition 


From  fundamental  probability  theory, 


E 


3  ■ 


ID 

I 


0  .•» 
I, 


Substituting  the  previously  computed  values  of  and  Ej  we  get 


0.824 
3  ""  0.838 


=  0.983 


E^lr)  .=  Effectiveness  in  detection  and  track  versus  range 


E^(r)  =  a'|c(0,  r)]  [b(30)]  C(30) 


=  -2  ^3] 


Cj(0,r) 


0  0 
0  C^iO,  r)  0 

0  C^iO.r) 


0 


'^11  '^12 

0  '^22  'i; 

0  0  d 
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Cj(30f 

02(30) 

€3(30) 

We  could  now  put  in  the  numerical  values  for 

the  elements  a.,  d. .  and  C.(30)  used  in  the  other  examples  and  for  each 
^  ^  J  ^ 

assumed  value  of  r  compute  a  value  of  E^.  We  shall  not  go  through  the 
details  here  but  will  show  the  type  of  results  obtained  in  the  next  section 
of  the  report. 


5.  2  Application  of  Model  Results 

It  was  stated  earlier  that  the  purpose  of  a  figure  of  merit  is  to 
provide  management  with  information  on  which  to  make  decisions.  The 
numerical  value,  when  compared  to  goals,  indicates  whether  or  not  cor- 
rective  action  is  necessary.  The  sensitivity  of  the  figure  of  merit  to  varia¬ 
tions  in  system  parameters  indicates  the  action  or  combination  of  actions 
to  be  taken. 


And  this  is  the  real  value  of  quantifying  system  effectiveness  -- 
the  ability  to  use  the  figures  of  merit  to  take  corrective  action.  The  figures 
of  merit  themselves  may  mean  little.  For  example,  we  get  0.983.  So 
what?  The  usefulness  of  the  figure  of  merit  comes  when  we  compare  it  to 
some  minimum  acceptable  requirement  or  other  criteria,  or  show  effec¬ 
tiveness  variation  with  variations  of  the  effectiveness  parameters.  Then 
we  can  make  a  decision.  First,  do  we  need  to  take  action?  Second,  what 
action  do  we  take  ?  Let  us  look  at  some  examples. 
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In  the  previous  sections  we  computed  the  overall  mission 
effectiveness  (E^)  to  be  0.824.  Let  us  assume  that  our  original  goal  was 
0.90  (90%).  We  are  short  of  the  goal,  so  what  can  be  done  to  reach  or 
exceed  it? 


First  a  qiick  glance  at  the  different  figures  of  merit  shows  that 
most  of  the  trouble  is  in  initial  target  detection,  since  the  effectiveness  in 
tracking,  given  target  detection  (E^)  is  0.983  and  effectiveness  in  detec¬ 
tion  alone  (E^^)  is  only  0.838.  Further,  examination  of  the  computations 
used  to  find  E^  shows  that  even  if  the  availability  were  perfect 

(a  t  “  “]) 

effectiveness  would  only  be  0.90  since. 


E^  =  1  0 


=  0.90  . 


■*> 

Thus  we  must  conclude  that  the  system  is  limited  by  Capability 
and  that  either  the  transmitter  power  must  be  increased  or  range  must  be 
sacrificed. 


Assume  now  that  the  system  is  in  the  Operational  Phase  and  that 
an  increase  in  power  would  be  expensive,  how  much  sacrifice  in  range  is 
involved?  Fortunately  the  equations  can  be  evaluated  rapidly  on  a  com¬ 
puter  and  Figure  4  shows  the  overall  effectiveness  (E^)  as  a  function  of 
the  transmitter  MTBF  for  several  values  of  range.  The  MTBF  of  the 
receiver,  etc. ,  combination  and  the  MTTR's  are  considered  fixed  at  the 
values  previously  used  From  the  curves  we  see  that  with  a  transmitter 
MTBF  of  40  hours  -•  the  value  previously  assumed  --  the  range  sacrifice 
for  0.90  effectivene.s  s  is  less  than  20  miles;  i.e.,  range  drops  from  200 
to  slightly  over  180  riiles.  This  might  be  considered  satisfactory  if 
money  for  improvement  was  hard  to  get. 

It  is  noted  also  in  looking  at  the  curves  of  Figure  4  that  the  curves 
rise  rather  rapidly  with  increasing  transmitter  MTBF  up  to  about  100  hours 
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and  then  increase  much  more  slowly.  It  is  also  known  that  transmitter  \  J 

failures  are  expensive  both  in  replacement  parts  costs  and  in  maintenance 
time.  It  is  clear  that  it  should  be  possible  to  cut  maintenance  effort 
roughly  in  half  by  increasing  this  MTBF.  What  improvement  in  range  for 
0.90  effectiveness  might  be  realized  at  the  same  time?  Figure  5  shows 
overall  effectiveness  as  a  function  of  range  for  a  transmitter  MTBF  of 
100  hours.  The  curve  crosses  the  90  percent  effectiveness  line  at  almost 
190  miles  range.  On  the  other  hand  it  increases  effectiveness  at  200 
miles  range  from  the  previous  0. 824  to  slightly  over  0.85.  The  obvious 
question  that  comes  to  mind  is:  How  much  is  this  improvement  in  MTBF 
worth?  If  it  can  be  done  for  the  savings  in  maintenance  costs  alone  it 
is  evidently  worthwhile.  If  it  costs  more  than  that  one  must  then  attempt 
to  evaluate,  also,  the  gain  in  effectiveness  at  fixed  range  or  the  improve¬ 
ment  in  range  sit  fixed  effectiveness.  These  evaluations  are  beyond  the 
scope  of  this  report,  but  guidelines  for  this  purpose  are  the  aim  of  Task 
Group  IV . 

Up  to. this  point  we  have  varied  only  two  of  the  explicit  "accountable 
factors"  in  our  model  --  the  failure  characteristics  of  the  transmitter  and  •*> 

the  range  requirement.  It  is  noteworthy  to  see  what  happens  when  trans¬ 
mitter  MTTR  is  varied.  Referring  to  Figure  6  it  can  be  seen  that  the  trans¬ 
mitter  MTTR  has  an  appreciable  effect  on  effectiveness  only  when  transmitter 
MTBF  is  below  about  100  hours.  It  can  also  be  noted  that  doubling  the  MTBF, 
say  from  40  hours  to  80  hours  has  the  same  effect  as  cutting  the  MTTR  in 
half.  These  two  alternatives  produce  the  same  effect  on  maintenar.ee  man¬ 
power  costs  but  not  on  replacement  part  costs.  Thus  in  a  choice  between 
increasing  MTBF  or  decreasing  MTTR,  increasing  the  MTBF  is  favored 
if  proportionate  changes  can  be  made  for  the  same  amount  of  money. 

Figure  6,  more  importantly  demonstrates,  as  noted  earlier, 
the  overriding  influence  of  the  range  requirement. 

Finally,  it  is  interesting  to  examine  what  improvement  is  actually 
realized  through  the  use  of  the  two  parallelled  transmitters.  In  Figure  7 
the  curve  labeled  "Configuration  1"  is  the  system  as  we  have  been 
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discussing  it;  i.e.,  with  the  paralleled  transmitters .  The  curve  labelled 
"Configuration  2."  is  the  system  with  a  single  transmitter.  At  the  longer 
ranges  the  single  transmitter  loses  effectiveness  due  to  its  lower  power 
and  the  loss  of  redundancy.  At  the  shorter  ranges  the  loss  is  due  only  to 
the  loss  of  redundancy.  Quite  evidently  there  is  an  appreciable  gain  in 
effectiveness  brought  about  by  this  change  in  configuration.  The  gain 
however  is  accompanied  by  appreciable  cost  due  to:  (1)  two  transmitters 
instead  of  one,  and  (2)  increased  maintenance  costs  to  maintain  two  trans¬ 
mitters  rather  than  one. 

Is  the  gain  in  effectiveness  produced  by  the  two  transmitters  worth 
the  cost?  While  this  question  is  beyond  the  scope  of  this  task  group  effort, 
a  brief  discussion  is,  nevertheless,  appropriate. 

Especially  with  a  defensive  weapon  system  the  "value"  of  the  mis¬ 
sion  is  essvmtially  negative;  i.e.,  to  prevent  or  minimize  loss.  Thus  it  is 
reasonable  to  assume  that  with  each  failure  of  the  system  under  critical  use 
conditions  some  average  loss,  either  in  dollars,  lives,  or  other  measures, 
can  be  assigned.  It  follows  then  that  a  system  that  has  half  the  failure  proba¬ 
bility  of  another  has  twice  the  "value,  "  and  consequently  we  should  compare 
such  systems  on  their  probability  of  failure  rhther  than  success.  This  means 
that  the  complement  of  the  effectiveness  figure  (1  -  E)  is  of  greater  interest 
for  comparisons. 


Following  the  above  rationale  and  returning  to  Figure  7,  we  note 
that  Configuration  2  has  a  probability  of  (1  -  0.59)  or  0.41  of  failure  to  carry 
out  its  mission  if  the  range  requirement  is  200  miles  and  Configuration  1  has 
a  similar  probability  of  approximately  0.18.  Thus  the  "value"  of  system  1 
is 


0.41 

wrn 


=^2.3 


I 

'<i> 


times  that  of  system  2.  In  the  regions  of  lower  range  this  figure  rises  to  3..0. 
Certainly  then  if  system.  2  was  considered  to  have  value  commensurate  with 
its  cost,  system  1  would  be  a  better  buy  if  the  cost  were  less  than  2.3  to  3.0 
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times  the  cost  of  system  2,  Since  the  cost  of  the  extra  transmitter  should 
not  be  this  great  and  neither  should  the  relative  increase  in  maintenance 
costs,  the  redundancy  is  definitely  justified. 

5.  3  Simulation  Method 

The  application  of  Monte  Carlo  simulation  methods  to  the  previous 
example  can  be  amply  demonstrated  by  restricting  attention  to  initial  tar¬ 
get  detection  at  maximum  range.  This  restriction  requires  only  that  we 
estimate  the  availability  and  capability  vector  elements;  and  in  order  to 
show  the  power  of  the  simulation  method,  interdependence  of  system  per¬ 
formance  parameters  will  be  taken  into  consideration  in  estimating  the 
capability  elements. 

Programming  a  digital  computer  to  perform  Monte  Carlo  simula¬ 
tion  requires  three  basic  steps: 

(1)  Mathematical  formulation; 

(2)  Flow  diagraming;  and 

(3)  Coding. 

Mathematical  formulation  suitable  for  digital  computers  requires 
the  problem  to  be  stated  in  terms  of  numbers  and  logical  steps,  not  equations. 
Thus  it  is  necessary  to  reduce  all  equations  to  an  equivalent  set  of  simple 
logical  steps  involving  only  the  basic  arithmetic  operations  of  addition,  sub¬ 
traction,  division  and  multiplication. 

Flow  diagramming  consists  of  block  diagramming,  in  step-by- 
step  order,  every  operation  the  computer  must  perform. 

Coding  consists  of  translating  the  flov»?  diagram  into  computer 

language. 

In  this  illustration  only  the  mathematical  formulation  and  flow 
diagramming  will  be  shown. 

5.  3.  1  Availability  Vector  Elements 

Assuming  that  the  time  required  to  repair  a  subsystem 
is  independent  of  the  time-between-failures,  the  availability  of  that 
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subsystem  can  be  estimated  by  operating  it  a  long  time  and  then  performing 
the  following  sequence  of  calculations:  (1)  add  the  total  time  spent  in  an 
operating  state;  (2)  add  the  time  spent  in  a  repair  state;  (3)  add  the  times 
found  in  (1)  and  (2);  and  (4)  divide  the  results  of  (1)  by  those  of  (3),  The 
foregoing  steps  could  be  written  as  an  equation,  but  as  stated  they  are  much 
closer  to  the  form  needed  for  computer  simulation. 

If  we  had  actual  measured  numbers  of  the  type  described 
above  v/e  could  proceed  directly  to  an  estimate  of  the  subsystem  availability. 
Lacking  the  specific  numbers,  they  can  be  generated  in  the  computer  from 

(1)  knowledge  of  the  manner  in  which  the  time  elements  are  distributed; 

(2)  knowledge  of  the  parameters  of  the  distribution;  and  (3)  a  table  of  random 
numbers  in  the  range  0  to  1.  For  example,  it  is  commonly  observed  that 
times-between-failures  are  distributed  exponentially  and  times-required- 
to- repair  are  distributed  log-normally.  In  the  first  case,  mean-time- 
between- failures  is  the  only  parameter  needed  to  completely  define  the 
distribution.  In  the  secord  case,  the  mean-time  and  some  measure  of 
dispersion  are  the  required  parameters.  The  tables  of  random  numbers 
are  universally  available.  Details  of  the  computer  subroutines  for  calcu¬ 
lating  the  random  time  increments  will  not  be  given  here  since  they  are 
generally  available. 

Having  described  methods  for  generating  times-between- 
failure  and  times-to- repair  that  are  random  in  length,  the  detailed  simu¬ 
lation  of  the  system  behavior  can  be  flow  diagrammed  as  shown  in  Figure  8. 
The  detailed  steps  and  symbols  are  described  in  TABLE  IIL 

5.3.2  Capability  Vector  Elements 

To  find  the  probability  of  target  detection  at  any  range, 
or  at  a  random  point  in  time,  one  simulates  the  target,  synthetically 
exercises  the  system,  and  notes  whether  the  target  is  detected  or  not.  The 
ratio  of  the  total  number  of  successful  target  detecticms  to  the  total  num¬ 
ber  of  trials  is  an  estimate  of  the  probability  of  target  detection.  This 
probability  is  a  function  of: 
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(1)  The  noise  figure  of  the  receiver 

(Z)  The  power  transmitted  from  the  antenna 

(3)  The  performance  of  target  detection  cir¬ 
cuits  in  the  receiver 

(4)  The  operation  of  the  display  device 

(5)  The  competence  of  the  human  operator. 

It  is  assumed  that  receiver  and  transmitter  performance 
are  dependent  on  each  other;  i.  e. ,  satisfactory  performance  can  still  be 
obtained  if  the  transmitter  power  is  high  and  the  receiver  sensitivity  is  low, 
or  the  receiver  sensitivity  is  high  and  the  transmitter  power  low. 


Specifically,  we  assume  that  the  ratio  of  receiver  out¬ 
put  signal  power  Pg  to  receiver  output  noise  power  is  given  by: 


where 

S 


Pr 


where 

a 

Pt 

r 


Ps  s2Pr 


signal  gain  of  receiver  in  volts  output/ volt  input 
radar  return  signal  power  at  antenna. 

The  radar  return  signal  power  is  given  by: 


constant,  involving  target  cross  section  and  other  factors 
total  radar  transmitter  power 
range  to  target. 


Substituting  this  value  of  Pr  in  the  former  equation  gives: 


Pa  -aSZPT 
Pn  Pnr4 


(Eq.  6) 


In  any  given  attempt  to  detect  the  target,  the  ratio 

Pg/P^  must  exceed  a  given  value  j_  which  depends  upon  the  target 

detection  circuits  of  the  receiver.  Whether  or  not  j  will  be  equalled 

or  exceeded  depends  upon  the  probatdlity  distributions  of  transmitter 

power  P(P^),  receiver  signal  gain  P(S),  and  background  noise  power 

P(P  ).  The  first  two  distributions  are  determined  from  direct  measure 
n 

ment  and  observation  of  the  transmitter  and  receiver.  In  general,  they 
will  have  the  form  illustrated  in  Figures  9  and  10. 
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Program 


et  j  = 


Set  Number  of 
Cycles  of 
Computation 

i  =  N 
max 


Select  t'^ 


y?  =  X'?  + 

1  1  r. 


Select  Subsystem 
(i.  e. ,  Advance  j) 


Exit  to 
Output 
Routine 


j-  5  =  0  )-Wi  -N  =  0j 


J  ^ 
a*'  = - - 


—  MHAdv.  j 


j-5  =  0 


store 


ompute; 
A  — ■►store  {~>|  Store 
-  A2,  A^ 


FIGURE  8 


FLOW  CHART  FOR  AVAILABILITY  SIMULATION 
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TABLE  ni 

STEP-BY-STEP  PROGRAM 


j  =  1  =  transmlttur  number  one 
j  r.  z  =  transmitter  number  two 
j  =  3  =  iintcnna 
j  =  4  =  roci'ivor 
j  =  5  =  display. 

The  subscript  i  denotes  how  many  random  times 
have  been  taken;  i.  e. ,  how  many  cycles  of  calcu> 
latlon  have  been  performed. 

The  sequence  of  steps  In  this  simulation  In  the  1^ 
cycle  arc  as  follows: 

(1)  Select  subsystem  J 

(2)  Select  the  i^^  random  time  to  failure  for 

the  subsystem  t/ 

‘  I 

(3)  Add  this  time  to  failure  to  the  sum  of  all 
previous  times  to  failure  and  the  sum  of  all 
previous  times  to  repair  Xj 


x.^  s  ti  3- 


i-1 


where 


i-1 


i-1 


'i-.-Z  ■1*1  -i. 


m=l 


m=  1 


(4)  Store  x.'* 

(5)  Add  this  t:.me  to  failure  to  the  sum  of  all 
previous  times  to  failure 


“i-l 


=  f* 


where 


“l  “l-l  ^1 


i-1  , 
tJ  =  ti 
“i-1  ^  ^r 

m=l 


(6)  Store  f* 


1 


th 


(7)  Select  time  to  repair  for  i  failure  for  the 
j  "  subsystem  t^  •*:  i.  c. ,  select  a  random 
number  from  the  random  number  tabic, 

enter  the  function  generator  subroutine  and  cal¬ 
culate  (or  look  up)  time  corresponding  to  the 
repair  rate  p'*  of  the  subsystem. 

(8)  Add  this  time  to  repair  to  the  sum  of  all 
previous  times  to  failure  and  the  sum  of  all 
previous  times  to  repair  x.-*: 

yj^  a  Xj^  +  t|, 


(9)  Store 

(10)  Answer  the  question:  Is  the  current  value 

of  j  equal  to  the  total  number  of  nubsystemn? 

/  ’ 
j  -  5  =  0 

(11)  If  the  answer  to  Step  (10)  is  no,  select  the 
next  Bubsysteni;  1.  e. ,  advance  J  by  one  and 
re-enter  routine  at  Step  (1). 

(12)  If  the  answer  to ‘Step  (10)  is  yes,  set  j  =  1 
and  answer  the  question:  Is  the  current 
value  of  1  equal  to  the  planned  number  of 
cycles  of  computation  N? 

7 

1  r  N  =  0 

(13)  If  the  answer  to  Step  ( 12)  is  yes,  enter 
compute  availability  routine  (Step  (15)). 

(14)  If  the  answer  to  .Step  (12)  is  no,  advance  i 
by  one  and  re-enter  routine  at  Step  (1). 

(15)  Compute  availability  of  subsystem 

f  j 


J 


(16)  Store  S'* 

(17)  Answer  the  question:  Is  the  current  value 
of  ^  eqnal  to  the  number  of  subsystems? 

7 

j  -  5  =  0 

(18)  If  the  answer  is  no,  .uh.ince  j  to  j  3  1  and 
re-enter  availability  computation  at  Step  (15). 

(19)  If  the  answer  is  yes,  compute  the  availability 
vector  component  Aj 

aJ 
j=l 

(20)  Store  Aj 

(21)  Compute  the  availability  vector  component  A., 

a  r  .  n  2vx,l  .V  5  5  ^ 

A^  =  la'  (1-a  )  +  (1-a')  a  J  a  a  a 

(22)  Store  A2 

(23)  Compute  the  availability  vector,  component  A^ 

(24)  Store  A, 


Aj  =  X 


Aj  =  1  -  (Aj  4  A^) 


(25)  Exit  to  output  routine. 


59 


Probability  that  Actual  Receiver 
Gain  is  Eiqual  to  or  Less  than  S 


FIGURE  9 


CUMULATIVE  PROBABILITY  DISTRIBUTION  OF 
TRANSMITTER  POWER  VARIATIONS 


CUMULATIVE  PROBABILITY  DISTRIBUTION  OF 
RECEIVER  GAIN  VARIATIONS 


The  probability  that  the  noise  power  will  be  equal  or 
less  than  at  a  random  point  in  time  is  also  given  by  measurement  and 
observation.  It  is  given  by  a  curve  similar  to  that  of  Figure  11. 


CUMULATIVE  PROBABILITY  DISTRIBUTION  OF 
RECEIVER  OUTPUT  NOISE  POWER 

These  three  empirical  distributions,  in  conjunction  with 
Equation  6  will  permit  a  simulation  of  the  capability  vector.  The  first 
step  is  to  simulate  random  values  of  S,  P^,  and  P^.  This  is  done  in  pre¬ 
cisely  the  same  manner  described  earlier  for  times  to  failure  and  times  to 
repair.  From  a  table  of  random  numbers  we  select  a  value  which  we 
identify  as  one  of  the  probilities  P^(S),  P^(Prp),  or  P^(P^).  From  the  above 
curves  we  match  these  P.  to  the  corresponding  signal  gain  S^,  transmitter 
power  PXj^j  noise  power  P^^^,  and  calculate  Equation  5-1.  If  the  result¬ 
ant  number  is  equal  to  or  greater  than  some  preselected  limit  of  signal  to 
noise  ratio  (often  taken  as  1.  0)  we  have  scored  a  successful  target  detec¬ 
tion;  if  not,  we  have  scored  a  failure.  This  process  of  selecting  the  and 
P^  is  continued,  remembering  the  successes  and  failures,  until  a  satisfac¬ 
tory  estimate  of  the  probability  of  detection;  P^  can  be  made  by  computing, 
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^  _  total  number  of  successes 
■^d  "  total  number  of  trials 

It  should  be  noted  that  this  simulation  is  done  for  each  availability  state. 
For  state  Aj,  which  has  two  transmitters  in  operation,  we  calculate 


Pp  S^[p„(l)+  P  (2) 

_  Q 1= _ :  . 


For  state  A^,  which  has  only  one  transmitter  in  operation,  we  calculate 


7“  =  a - 5- 

N  V 


P  S^P  ^ 

R  T 

p—  =  a - ^ 

N  V 

where  P,p^^^  is  the  power  transmitted  from  transmitter  number  one  and 
is  the  power  transmitted  from  transmitter  number  two.  Since  we  have  not 
differentiated  between  transmitters  in  any  way,  either  of  the  last  two 
equations  may  be  used  for  state  A^.  This  sequence  of  steps  is  briefly  out¬ 
lined  in  Figure  12. 

5.  3.  3  Effectiveness 

Since  we  have  at  this  point  simulated  all  that  is  required 
to  calculate  the  particular  figure  of  merit,  "the  probability  of  target 
detection  at  maximum  range,  "  we  compute 

E  =  ■A-jC|+  ^2^2}  ^3  “ 

This  step  is  shown  in  Figure  12. 

5.  3.  4  Simulation  Accuracy 

If  the  computer  repeated  the  above  routines  an  infinite 
number  of  timesj  the  analytic  results  and  the  simulation  results  would  be 
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FIGUFvE  12.  FLOW  CHART  FOR  CAPABILITY  SIMULATION 
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identical.  However,  reasonably  good  approximations  can  be  obtained  using 
small  samples.  For  example,  when  the  machine  output  is  a  cumulative 
distribution  function  and  when  the  sample  size  increases  to  infinity,  then 
with  probability  one,  the  maximum  deviation  between  Fj^’x)  and  F{x)  tends 
toward  zero. 


Pr 


F(x)l>§ 


-♦0  as  N  •+  ® 


Pr 


sup 

-  00  <  X  <  » 


as  N  . 


Thus,  empirical  distribution  functions  derived  from  a 
Monte  Carlo  Simulation  will  uniformly  approximate  the  shape  of  the  true 
distribution  function.  Also,  when  the  distribution  function  F{x)  is  a  con¬ 
tinuous  function,  the  empirical  cumulative  distribution  for  N  observations 
Fj^(x)  will  uniformly  approximate  the  true  distribution  with  an  accuracy  of 
the  order  of  1//N,  which  is  one  consequence  of  the  Kolmogorov-Smirnov 
goodness -of-fit  test. 


The  statistic 


max 

-  00  <X  <00 


Fj^(x)  -  F(x)  I 


measures  the  greatest  absolute  difference  between  the  true  cumulative 
density  function  F(x)  and  the  empirical  cumulative  density  function  Fj^(x). 

is  a  "distribution  free"  statistic  ^{i.  e.,  its  sampling  distribution  is 
independent  of  the  underlying  distribution  F(x)^  and  its  distribution  is  known. 
This  is  useful  in  answering  the  following  question:  How  large  should  N  be 
chosen  so  that  there  is  only  a  small  preassigned  probability  that  (Dj^  =5)? 
Symbolically. 

1=  ^ 


If  a  =  0.  10,  0.  05,  and  0.  01,  respectively,  where  a  is  the  acceptable  risk 
(type  I  error)  of  test,  itcanbe  shown  that  the  approximate  sample  sizes  N  are 
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If  €  =  0.  05  and  a 
If  e  =  0.  05  and  q' 
If  e  =0.  05  and  a 


0.  10, 

then 

N 

0.05, 

then 

N 

0.01, 

then 

N 
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=  595, 
=  740, 
=  1060. 
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APPENDIX  I 


MATHEMATICAL  DETAILS  OF  TUTORIAL  EXAMPLE 

1.  Computation  of  Availability  “(lector  Elements 

Let  =  availability  of  each  transmitter 

=  availability  of  comppsite  antenna,  receiver,  display,  and 
synchronizer. 

Then  from  Section  4.  2: 

^  _  MTBF  _  40  _ 

®T  "  MT'BF  “MTTR  "  '40  +  T  "  * 

"r  " 

Also  from  Section  4.  2: 

a^  =  o^rj,ofrj,0!^  =  =  (•  909)^.  99  =  .  818 


^2  " 


I’a^ll-a^)  +  (l-Q'^)Qr^jQrj^ 


=  2a^(l-a^)(X^  =  2  X  .  909(1-.  909).  99  =  .  149 


a^  =  1  -  (a^  +  a^)  =  1  -  (.  818  +  .  149)  =  .  033. 

2.  Computation  of  Dependability  Matrix  Elements 

In  Section  5.  1.8.2,  X  and  were  defined  as  the  failure  rates  of  a 

i  K. 

transmitter  and  the  composite  receiver,  antenna,  display  and  synchronizer, 
respectively.  The  corresponding  unit  reliabilities  were  given  as: 

R^(30  min)  =  exp  =  exp  025  x  .  5|  =  .  988 


Rj^(30  min)  =  exp  =  exp  010  x  .  sj  =  . 


995 


In  this  example  d^^^^  is  the  probability  that  all  units  begin  the  mission  and 
continue  to  operate  throughout.  Thus, 
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dll  " 


=  exp  -  -[(2  X  .  025  +  .  010)a5}  =  0.  971 

Similarly  di2  is  the  probability  that  all  units  begin  the  mission  but  one 
transmitter  fails  during  the  mission;  all  other  units  operate  throughout. 
Thus, 

^12  ”  (l"R.p)R.pRl^ 

=  2(l-R^)R^Rl^ 

=  2  exp  -  |(X^  +  Xi^)tj-  -  2  exp  -  {(2X^  + 

=  2  exp  -  |(.  025  +  .  010)0.  5}  -  2  exp  -  -[(2  x  .  025  +  .  010)0.  5o} 

=  0.  024. 

The  system  beginning  the  mission  in  state  1  must  certainly  complete  it 
in  one  of  the  three  states  1,  2,  or  3  and  so, 

‘^IS  "  ^  "  ^‘^ll  ^^12^ 

=  1  -(  0.  971  +  0.  024)  =  0.  005 

In  similar  fashion  the  other  elements  are  found  to  be: 

^22  = 

=  exp  -  |(.  025  +  .  010)0.  5}  =  .  982 
^23  =  1-^22  =  °-°^® 

and 

^^33  " 
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Computation  of  Capability  P>elatj.ons 
In  Section  5.  1.  8.  3. 1  the  proportion 


O 


P  r  ^ 

In(l-p^)  ^2 

ln(l-p2) 

2 

was  presented  without  proof.  In  this  expression 

p^  =  the  probability  of  detection  in  case  i, 

P^  =  transmitter  power  in  case  i,  and 
i 

r^  =  range  in  case  i. 

The  expression  is  derived  as  follows: 

In  the  literature  (Reference  17)  it  is  shown  that  the  probability  density 
function  of  instantaneous  thermal  agitation  noise  power  is: 

f(P  )  =  — L- exp  <-  - 0<P  <» 

where 

P^  is  instantaneous  noise  power 

and 

C^  is  the  r.  m.  s.  noise  amplitude. 

If  we  assume  that  a  signal  will  be  detected  if  the  signal  power  at  the 

receiver  input  P^  is  equal  to  or  greater  than  the  noise  power,  then  the  pro 
K. 

bability  of  detection  is 

exp 

-0 

which  integrates  to  giv. 


p(Pn<PR)  = 


2C 
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p(Pj^  <  Pr)  =  1  -  exp 


ln(l-p)  = 


From  the  "radar  equation,  "  it  is  known  that  the  signal  power  returned 


to  the  receiver,  Pj^,  is  given  by 


aP^ 

r 


where 


P,j,  =  transmitter  power 
r  =  range 

a  =  a  constant  involving  target  cross  section  and  other  factors^ 
Substituting  this  equation  in  the  probability  equation,  gives 


In(l-p)  =  - 


2C  r 


And  since  we  assume  that  p  has  been  determined  for  one  set  of  values  of 
power  and  range  other  relations  can  be  found  from  the  ratio. 


In(l-p^) 


P  Y 


Z'  P 


'^2  ’^l 


This  expression  can  be  solved  for  p^  as  follows 


P  r 
T  1 

In(l-P2)  = - - - ^  In(l-p^) 
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r 


1  -  p  =  exp  ^ 


^2 


P  r 

2 


,  1 
^  ln(l-Pj^)V 


4 

^2 


J 


P2  -  1  -  exp  /  - ^  In(l-Pj)  > 


Pt/2 


J 


If,  as  assumec^  in  the  example,  p^  is  evaluated  to  be  0,  9  at  full  tranS' 
mitter  power  output  and  200  mile  range,  then 

ln(l-0.  9)  =  -  2. 3 

and 


p~  =  1  -  exp 


j 


P™  (200) 

■^2  i 
A  -^-3  4 

Pnn  r. 


L 


Tj  ^2 


where 


P„  is  the  reference  power. 

For  state  1  of  the  system  P„  =  P_  and  thus, 

^2  n 


Cj(0)  =  1  -  exp 


In  system  state  2.P  =-ip„  and 

^2  1 


r 


C2(0)  =  l-exp/  -  2.  3  ) 


2r. 
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AFh'JiiNJJlX  II 


SUMMARY  OF  THE  AIRBORNE  AVIONICS  SYSTEM  EXAMPLE 
(Complete  example  included  in  Volume  III.) 

The  purpose  of  this  example  is  to  demonstrate  how  the  effectiveness 
evaluation  techniques  proposed  by  Task  Group  11  and  discussed  earlier  in 
this  report,  may  be  applied  to  the  Avionics  system  of  a  tactical  fighter - 
bomber  aircraft.  The  example  considers  only  the  "bombing"  fujpction. 
Similar  analyses  could  be  made  for  its.  "fighter,  "  "ground  support,  "  etc. , 
functions. 

It  is  assumed  that  the  effectiveness  evaluation  is  being  made  during  the 
program  definition  phase  of  system  life.  Similar  evaluations  in  the  real 
world  would  also  be  necessary  for  system  configurations  established  during 
the  acquisition  and  operational  phases.  A  major  consideration  of  the  pro¬ 
gram  definition  phase  is  "force  structure,  "  i.  e.  ,  the  number  of  system 
(aircraft)  required  to  accomplish  a  specific  mission.  The  example  illus¬ 
trates  how  the  results  of  the  effectiveness  evaluation  aid  in  making  trade-off 
decisions. 

The  basic  model  proposed  by  Task  Group  II;  i.  e.  ,  E  =  A  J  is 
applied  in  this  example. 

1.  Mission  Definition 

The  system  is  evaluated  as  a  tactical  weapon.  The  aircraft  is  con¬ 
sidered  to  be  deployed  at  an  advanced  base  in  the  theater  of  operations,  and 
is  called  upon  to  bomb  tactical  targets  in  enemy  territory.  It  is  assumed 
that  no  enemy  offensive  action  will  be  mounted  against  the  advanced  base. 

At  any  random  time  when  an  execution  order  is  received,  the  aircraft 
shall  take  off  immediately,  receive  a  target  assignment,  proceed  to  target 
area,  deliver  weapon  within  500  feet  of  target,  and  return  to  assigned 
operation  base. 

2.  System  Description 

The  system  consists  of  three  major  subsystems  which  are,  where 


72 


appropriate,  sub-divided  into  equipments; 

a.  Fire  control  Subsystem 

b.  Doppler  Navigator 

c.  Communication-Identification-Navigation  (CIN) 

Further  analysis  of  these  subsystems  and  their  functions  is  made  in  the 

example.  These  analyses  include  a  block  diagram,  time  line  analysis  of 
mission,  list  of  pertinent  physical  factors,  etc.  . 

3.  Specification  of  Figures -of-Merit 

The  major  figure -of -merit  is  the  probability  that  the  mission,  as 
defined,  will  be  accomplished. 

In  addition,  the  probabilities  of  accomplishing  each  of  several  sub¬ 
functions  are  regarded  as  appropriate  figures -of-merit  and  are  evaluated. 

4.  Identification  of  Accountable  Factors 

Accountable  factors  are  considered  in  detail  in  the  example.  Major 
categories  are: 

a.  Operational  conditions 

b.  Support  situation 

c.  Data  Constraints  and  sources. 

At  this  early  phase  of  program  life,  generic  failure  rate  information  is 
of  prime  importance  since  contractor  bench  tests  from  which  supplementary 
data  may  be  obtained,  have  not  yet  been  made.  Several  sources  for 
obtaining  maintainability  and  generic  failure  rate  information  are  referenced 
in  the  example. 

5.  Model  Construction 

a.  Delineation  of  Mission  Outcomes 

The  several  possible  outcomes  of  the  bombing  mission  are  discussed  in 
the  example;  e.  g.  , 

(1)  Mission  accomplished  exactly  as  defined,  or 

(2)  Aircraft  delayed  in  take-off,  or 
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(3)  Aircraft  does  not  deliver  bomb  within  500  feet 
of  target. 

b.  Delineation  of  System  States 

The  example  time  line  diagram  shows  the  three  possible  delivery- 
modes  and  indicates  the  equipments  required  during  various  portions  of 
the  mission  cycle  for  each  mode.  The  probabilities  of  accomplishing  the 
mission  in  each  mode  are  evaluated  by  estimating  and  combining  the  proba¬ 
bilities  of  each  sub -function;  e.  g.  ,  communication,  navigation,  etc..  The 
probability  of  performance  of  each  sub-function  is  determined  through 
consideration  of  the  equipment! s)  which  accomplish  the  function  and  their 
states.  Only  two  states  of  each  equipment  are  considered,  viz.  ,  operative 
and  failed. 


Some  sub-functions  are  accomplished  by  redundant  equipments.  There¬ 
fore  it  is  nec  isary  to  determine  the  probabilities  of  performance  of  these 
sub-functions  by  evaluating  all  combinations  of  redundant  equipments  in 
their  respective  operative  and  failed  states. 

c.  System  Model 

The  basic  model  expresses  the  system  effectiveness  as  the  sum  of  the 
products  of  the  effectiveness  values  for  each  mode  of  weapon  delivery  and 
the  probability  of  em,ploying  each  mode;  i.  e. , 


E  =  Pj  + 


^2  ^2  ^3  ^3 


where 


E  =  system  effectiveness. 

Ej^,  E^,  E^  are  effectiveness  figures  of  modes  one  (1), 

■^two  (2),  and  three  (3)  respectively,  and 
Pp  P^,  P^  are  the  probabilities  that  respective  mission 
modes  will  be  used. 

The  effectiveness  for  each  mode  is  expressed  as  the  product  of  the 
effectiveness  figures  for  the  several  sub-functions  required  during  a 
mission  employing  this  delivery  mode;  i.  e.  , 
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where 


=  effectiveness  of  mission  mode  one  (1)  and 

ttE.  =  product  of  effectiveness  figures  of  all  sub-functions 
required  for  mission  accomplishment  in  mode  one(l). 

The  effectiveness  for  each  sub-fimction  is  expressed  as  the  product  of 
availability  (K),  dependability  and  capability  for  each  sub-function; 

i.  e.  , 

Ei  =  S  [d]Z3. 

6.  Data  Acquisition 

a.  Specify  Data  Elements 

Elements  of  data  from  which  the  avionics  equipment  failure  rates  and 
repair  rates  can  be  computed  are  identified  for  collection. 

b.  Specify  Test  Methodology 

Test  methodology  for  producing  failure  rate  and  repair  rate  data  are 
identified. 

c.  Specify  Data  Collection  System 

Data  collection  systems  for  obtaining  both  operational  and  test  data 

are  identified. 

7.  Parameter  Estimation 

Values  of  individual  equipments  mean-time -between-failure  (t^)  and 
mean-time -to-repair  (t^)  are  assumed  for  this  example. 

8.  Model  Exercise 

a.  Availability 

The  readiness  figure  for  each  equipment  is  calculated  from  the 
equation, 

P 

S,  “  t^  +  t 
1  f  r 
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where 


P,;,  is  the  probability  of  being  in  an  operative  state. 

The  probability  that  the  equipment  is  not  ready;  i.  e. ,  is  in  state  2 
(failed),  is 


=  1  -  P. 


The  availability  for  non-redundant  equipments  is  then: 


where 


r 

H 

o 

Pr,  Po 

1  p. 

Si  Sj 

0  ^2 

- 

=  availability  vector  of  equipment  (i) 

P|^  =  probability  of  aircraft  launch  given  equipment  (i) 

1  is  in  an  operative  state 

P|^  =  probability  of  aircraft  la’'nch  given  equipment  (i) 

2  is  in  a  failed  state. 


The  availability  vector  for  redundant  equipments  is: 


where 


X.  = 
1 


P  P 

Si  S2 


o 


Pj^  =  probability  of  aircraft  launch  given  the  redundant 
n  equipments  required  to  accomplish  sub-function  (i) 
are  in  state  Pg  which  is  a  combination  of  operative 
and  failed  n  states  of  the  redundant  equipments. 
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b.  Dependability 


( 


The  dependabilities  of  non-redu  idant  equipments  are  computed  from 
the  expression: 

where: 

=  dependability  matrix  of  equipment  (i) 

dfi  =  probability  of  equipment  (i)  completing  its  mission 
in  an  operative  state  giver  that  it  started  in  an 
operative  state 

^12  ~  probability  of  equipment  (i)  failing  to  accomplish  its 

mission  given  that  it  started  in  an  operative  state 

^21  ”  probability  of  equipment  (i)  to  accomplish  its  mission 

in  an  operative  state  given  that  it  started  in  a  failed 
state 

^22  ”  probability  of  equipment  (i)  failing  to  complete  its 

mission  given  that  it  started  in  a  failed  state. 

The  dependability  matrix  of  redundant  equipments  is  computed  from: 


D 


^21  *^22 


where: 


^11  ^12  •  •  •  ^n 
'^21*  *  •  •  ^2n 


!  d  ,  .  .  ,  .  d 

I  nl  ] 

L 


dfi,  dj^2>  •  •  •  =  probability  of  mission  accomplishment 

for  various  combinations  of  operative 
and  failed  states  of  equipments  required 
to  accomplish  sub -functions  (i)  . 

c.  Capability 


The  capability  vector  for  non-redundant  equipments  is: 
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where: 


-  capability  vector  of  equipment  (i) 

=  capability  of  equipment  (i)  given  it  is  in  an 
operative  condition  (state  1) 

-  capability  of  equipment  (i)  given  it  is  in  a 
failed  condition  (state  2) . 

For  redundant  equipments,  the  capability  vectors  are: 


1 

where 

Cj,  C21  ...  =  capabilities  for  various  combinations  of 

operative  and  failed  states  of  equipments 
required  to  accomplish  sub-function  (i)- 

d.  Calculation  of  Effectiveness 

The  effectiveness  for  each  sub-function  is  computed  from  the 
expression  for  the  basic  model;  i.  e. , 


The  effectiveness  for  each  mission  mode  is  then  computed  from: 

^(mode  one)  ”  ^^i(mode  one) 

^(mode  two)  =  ^^i(mode  two) 

■^(mode  three)  =  ^^i(mode  three)  . 

The  single,  overall  system  effectiveness  is  finally  obtained  from: 

E  =  E,  P,  +  E.  P.  +  E-  P-  . 

11  2  2  3  3 
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APPENDIX  III 


SUMMARY  OF  THE  ICBM  FLEET  EXAMPLE 
(Complete  example  included  in  Volume  III) 


It  is  the  specific  object  of  this  example  to  illustrate  the  analysis  of  an 
ICBM  fleet  in  terms  of  the  formal  mathematical  structure  adopted  by  Task 
Group  II  of  the  Weapon  System  Effectiveness  Industry  Advisory  Committee. 
Symbolically,  this  structure  is  given  by: 

E  =  X'  [d]  r 


where 

E  is  system  effectiveness 

A  is  the  availability  vector  and  X' is  its  transpose 

D  is  the  dependability  matrix 

ZT  is  the  capability  (performance)  vector. 


In  particular,  the  analysis  illustrates  the  usefulness  of  models  in 
assessing  the  im.pact  of  potential  system  alterations. 

1.  Mission  Definition 

The  general  requirements  of  this  hypothetical  system  may  be  stated  as 
follows: 


Any  missile  of  the  ICBM  fleet  should  be  ready  to  accept  a  launch 
directive  at  a  random  point  in  time,  or  at  an  arbitrary  time  after  an 
alarm  condition  has  been  established  at  a  random  point  in  time.  It  should 
then  launch  successfully  within  a  prescribed  reaction  time,  fly  a  ballistic 
trajectory,  penetrate  enemy  defenses,  arm  fuse,  impact  within  the  pre¬ 
scribed  target  area,  detonate  and  yield  as  planned  with  a  prescribed  proba¬ 
bility  of  target  destruction. 

Minimum  acceptable  and  objective  numerical  system  requirements  for 
availability,  countdown,  flight,  and  probability  of  kill  are  postulated  in  the 
form  of  an  SOR. 
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2.  System  Description 


The  system  chosen  for  this  example  is  an  ICBM  squadron  consisting  of 
nine  sites  with  one  missile  per  site.  It  is  assumed  that  a  missile  contains 
five  launch  critical  subsystems  and  a  site  contains  three  launch  critical 
subsystems  as  follows: 


a.  Each  missile  contains  the  following  launch  critical  subsystems: 


_ Subsystem 

Re-entry  Vehicle 

Guidance 

Autopilot 

Propulsion 

Structure 


Subsystem 

Designator 

A 

B 

C 

D 

E 


b.  Each  launch  facility  (site)  contains  the  following  launch  critical 


subsystems: 


Subsystem 


Subsystem 

Designator 


Overhead  door  F 

Air  conditioning  G 

Power  generation  H 

and  distribution 


Time  line  analyses  for  each  subsystem  are  postulated  andean 
example  time  line  for  the  checkout  of  an  existing  ICBM  is  given.  In 
addition,  the  reliability  functional  block  diagram  of  an  existing  ICBM  is 
given  as  an  illustration  of  the  degree  of  system  complexity  that  must  be 
treated  in  current  systems. 

3.  Specification  of  Figures  of  Merit 

Severed  figures  of  merit  of  varying  complexity  are  considered.  The 
principal  figure  of  merit  is  the  expected  number  of  targets  destroyed  per 
squadron  calculated  under  the  assumption  that  a  launch  directive  is  received 
at  a  random  point  in  time.  Subsidiary  figures  of  merit  are  the  relative 
ranking  by  mode  of  operation  of  subsystems  in  terms  of  availability  and 
reliability. 
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4.  Identification  of  Accountable  Factors 
a.  Define  Level  of  Accountability 

The  degree  of  accountability  (in  this  example)  places  the  least 
accountable  level  at  a  subsystem,  and  the  highest  accountable  level  at  a 
squadron. 

The  depth  of  detail  to  be  accounted  for  in  the  model  is  specified 
under  the  following  four  headings,  by  subsystem,  by  site,  and  by  squadron. 

(1)  Personnel 

(2)  Procedures 

(3)  Hardware 

(4)  Logistics. 

(a)  Personnel 

The  model  reflects  the  possibility  of  queuing  in  un¬ 
scheduled  maintenance  due  to  insufficient  personnel. 

The  model  does  not  explicitly  differentiate  procedural 
errors  from  human  errors. 


(b)  Procedures 

The  model  specifically  accounts  for  the  following  pro 

perties  of  a  test: 

•  test  coverage 

•  test  error 

•  false  alarm 

•  oversight 

•  test  duration 

•  on  alert 

•  off  alert 
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(c)  Hardware 


The  models  reflect  the  possibility  of  four  failure  stress 
levels  for  periodically  checked  subsystems  depending  upon. the  modes  of 
operation: 

_1  :Jert 

2  checkout  and/or  countdown 

3  flight 

4  (iemating  in  checkout 

The  model  also  reflects  the  possibility  of  inherently 
undetectable  failures. 

(d)  Logistics 

The  model  specifically  accounts  for  spares  provisioning 
under  tactical  launch  conditions.  Sparing  is  not  treated  during  the  pretacti- 
cal  situation. 

b.  Targeting  Policy 

A  squadron  is  targeted  on  three  objectives,  three  missiles  to  an 
objective. 

c.  Physical  Factors 

(1) The  launch  site  is  regarded  to  be  impervious  to  counter¬ 
measures  except  when  the  overhead  door  is  open.  (Consider  ground 
invulnerability  to  be  unity. ) 

(2)  For  the  class  of  target  considered,  the  warhead  exhibits  a  unity 
damage  function. 

(3)  The  cross  range  and  down  range  miss  distances  arising  from 
errors  of  the  guidance  system  are  normally  distributed  and  independent. 

(4)  The  probability  of  propellant  depletion  is  zero  for  the  target 
ranges  used. 

(5)  Under  tactical  launch  conditions  two  launch  attempts  may  be 
made,  since  each  site  stocks  sufficient  spares  to  repair  one  countdown 
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abort.  No  retargeting  capability  exists. 

(6)  The  reliability  and  performance  capability  of^the  communication 
system  is  unity. 

(7)  Penetration  probability  is  unity. 

d.  Personnel  Composition 

Each  squadron  is  supported  by  four  maintenance  crews.  A  crew 
works  an  eight -hour  shift  with  every  fourth  day  off.  During  emergency 
conditions  not  lasting  longer  than  one  week,  all  crews  may  be  put  on 
twelve-hour  duty,  two  crews  operating  simultaneously.  Maintenance 
equipment  is  redundant  to  this  extent.  A  full  crew  is  required  to  maintain, 
checkout,  and/or  repair  a  failed  missile  or  launch  facility.  Scheduled 
maintenance  does  not  create  queuing  problems. 

Each  launch  site  is  fully  manned  twenty-four  hours  a  day. 

5.  Model  Construction 

a.  Delineation  of  Mission  Outcomes 

The  following  mission  outcomes  are  treated  in  the  model. 

(1)  Total  failure  (full  target  survival) 

(a)  Not  ready  to  enter  countdown. 

(b)  Aborts  countdown. 

(c)  Catastrophic  failure  in  flight. 

(d)  No  yield. 

(Z)  Partial  failure  (or  success):  (incoinplete  target  destruction) 

(a)  Falls  Avide  of  target  with  proper  yield. 

(b)  Falls  on  target  with  low  yield. 

(3)  Total  success  (target  destroyed) 

b.  Delineation  of  iystem  States 

During  pretactical  conditions  each  of  the  subsystems  is  assumed 
to  occupy  any  one  of  the  following  seven  states: 
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(1)  "up"  (on  alert)  and  nonfailed 

(2)  "up"  and  failed  detectably 

(3)  "up"  and  failed  undetectably 

(4)  "down"  (in  checkout/repair)  and  nonfailed 

(5)  "down"  and  failed  detectably 

(6)  "down"  and  failed  undetectably 

(7)  "down"  and  ii^  rework  -  Time  Compliance  Technical  Order 

(TCTO). 


In  addition,  one  additional  state  is  considered  in  the  post  alarm 
environment  namely,  down  and  awaiting  repair  (queuing). 

c.  System  Model 

The  principal  model  is 

E  =X'  [d]  r 

where 

E  =  expected  number  of  targets  destroyed  per  squadron. 

A'  =  availability  vector 

=  probability  that  at  a  random  point  in  time  exactly  k  of  the 
9  sites  is  up  and  nonfailed  where  k  ranges  from  zero  to  9 
inclusive 

D  =  dependability  matrix 

=  array  of  transition  probabilities  giving  tiie  probabilities 
that  if  k  of  the  9  sites  are  available,  r  of  them  will  launch, 
fly,  impact  in  the  target  area,  and  detonate  as  planned; 
where  r  ranges  from  zero  to  k  inclusive  and  k  ranges  from 
zero  to  9  inclusive 

U  =  capability  vector 

=  probability  that  a  total  of  y  targets  will  be  destroyed;  given 
that  r  missiles  are  delivered  to  the  target  area  and  are 
detonated. 

d.  Subsystem  Models 

Subsystem  models  for  availability  and  reliability  are  developed  for 
each  of  the  eight  launch  critical  subsystems.  These  models  illustrate 
both  periodic  checkout  and  continuous  monitoring  maintenance  policies. 
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Dependent  chorkout  policies  are  considered,  as  well  as  the  properties  of 
test  coverage  and  accuracy. 

e.  Piece  Part  Models 

A  detailed  model  is  developed  for  the  re-entry  vehicle  illustrating 
model  development  at  a  level  less  than  the  subsystem  level. 

6.  Data  Acquisition 

a.  Specify  Data  Elements 

The  elements  of  data  required  to  estimate  the  model  parameters 
are  listed  and  compared  to  the  elements  of  data  currently  acquired  in  the 
SAC  U-8Z  and  U-86,  and  the  AFLC  AFM  66-1  data  collection  system. 
Deficiencies  are  noted. 

b.  Specify  Test  Methodology 

Test  methodology  is  discussed,  involving: 

(1)  normal  field  operation 

(2)  special  field  exercises 

(3)  special  nonfield  tests 

(4)  failure  analysis 

(5)  depot  data 

c.  Specify  Data  Collection  Systems 

As  cited  above,  the  SAC  U-82  jmd  U-86,  and  AFLC  AFM  66-1 
data  systems  are  discussed. 

7.  Parameter  Estimation 

A  series  of  maximum  likelihood  and  least  squares  estimators  are 
developed  in  terms  of  the  type  of  data  available.  Attention  is  given  to  the 
use  of  both  field  data  and  the  results  of  tear  down  failure  analyses. 

8.  Model  Exercise 

Point  estimates  of  the  availability,  reliability  (dependability),  and 
performance  (capability)  of  each  subsystem  are  made.  These  estimates 
a.re  combined  to  obtain  A',  rDl  ,  and^.  Finally,  these  measures  are 
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combined  to  obtain  a  point  estimate  of  E,  the  expected  number  of  targets 
destroyed  per  squadron. 

'■I.  Applications 

a.  Comparative  Systems  Analysis 

The  results  of  the  model  exercise  are  compared  to  the  SOR.  This 
comparison  indicates  that  the  minimum  acceptable  values  for  system  relia¬ 
bility  in  countdown  and  flight  are  met,  although  the  reliability  of  the  re-entry 
vehicle  is  clearly  susceptible  to  improvement.  The  true  availability  of  the 
system  is  woefully  lower  than  the  acceptable  minimum,  although  the  apparent 
availability  is  relatively  high.  The  per-unit  kill  probability  is  also  in  dras¬ 
tic  need  of  improvement. 

b.  Parameter  Variation  Studies 

Parameter  variation  studies  are  initiated  on  the  availability  and 
capability  factors  to  assess  the  potential  for  system  improvement.  It  is 
shown  that: 

(1)  improved  monitoring  and  increased  reliability  of  the  power 
generation  and  distribution  subsystem  in  conjunction  with 

(2)  a  drastic  shortening  of  the  times  between  scheduled  checkouts 
on  several  subsystems 

and 

(3)  an  increase  in  guidance  accuracy  by  a  factor  of  two  will  be 
required  to  achieve  minimum  acceptable  system  performance. 

The  question  of  costs,  schedules,  confidence  factors,  relative 
strategic  value  of  the  system,  and  technical  feasibility  of  accomplishing  the 
required  system  alterations  are  not  considered. 
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APPENDIX  IV 


SUMMARY  OF  THE  RADAR  SURVEILLANCE  SYSTEM  EXAMPLE 
(Complete  example  included  in  Volume  HI.) 

This  example  illustrates  for  this  type  of  defense  system  the 
effectiveness  prediction  techniques  discussed  earlier  in  this  document. 

The  tasks  required  to  evaluate  system  effectiveness  have  been  considered 
throughout  the  four  phases  of  system  life,  and  the  increasing  amount  of 
detail  which  is  necessary  as  the  system  evolves  is  shown.  A  particular 
feature  of  this  example  is  an  illustration  of  the  manner  in  which  the  total 
number  of  system  states  is  reducible  to  a  lesser  number  of  significant 
states . 

1.  Mission  Definition 

The  requirements  of  this  fictitious  system  are; 

a.  Detect  airborne  objects  in  the  surveillance  sector  at  a  range 
of  not  less  than  3,  000  nautical  miles  . 

b.  Identify  the  objects,  and  determine,  within  30  minutes  whether 
or  not  they  constitute  a  threat. 

2.  System  Description 

As  the  system  is  shown  to  evolve  through  the  four  program  phases, 
its  configuration  changes  with  design  improvements.  For  comparative 
purposes,  two  models  are  constructed  and  carried  through  the  example. 
The  first  depicts  system  configuration  during  the  program  definition  phase. 
The  second  depicts  system  configuration  at  the  end  of  the  acquisition 
phase  and  beginning  of  the  operational  phase.  The  model  pertaining  to 
the  acquisition  phase  configuration  is  omitted  for  the  sake  of  brevity. 

a.  Program  Definition  Phase 

It  is  decided  during  the  program  definition  phase  that  three  radar 
equipments  will  be  required,  each  providing  surveillance  of  a  specified 
sector.  Associateu  functions  necessary  to  complete  the  surveillance 
requirement  are  identified  as  follows: 
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(1)  A  data  link  function  for  each  radar  equipment  to 
transfer  radar  data  to  a  computational  center; 

(2)  A  computer  function  to  store  input  data  and  to  pre¬ 
dict  impact  areas;  (the  single  computer  shall  serve 
all  radar  equipments); 

(3)  Three  communication  functions,  each  of  which  shall 
convey  data  from  its  associated  radar  to  its  data 
processor; 

(4)  Three  data  processors  and  three  displays  to  present 
data  from  associated  radar  to  decision  maker;  and, 

(5)  Necessary  prime  power  to  support  each  of  the  three 
subsystems . 

b.  Operational  Phase 


At  this  point  of  system  life  the  example  system  configuration  is 
firmed-up  and  is  now  defined  to  be: 

(1)  Three  radar  equipments,  each  of  which  shall  provide 
surveillance  of  a  selected  sector.  Any  of  the  radar 
equipments  shall  be  capable  of  operating  with  any  of 
the  three  antennas.  Switching  shall  be  possible  in 
less  than  three  minutes  for  the  transmitters  and  in 
less  than  1.5  minutes  for  the  receivers; 

A  spare  transmitter  shall  be  provided  which  can  be 
switched  into  any  of  the  three  equipments  in  le'ss 
than  three  minutes. 

(2)  Two  data  link  subsystems,  either  of  which  shall  be 
completely  capable  of  handling  all  radar  data. 

(3)  Two  storage  and  computing  subsystems,  either  of 
which  shall  be  completely  capable  of  storing  all  input 
data  and  predicting  impact  area. 

(4)  Three  communications  subsystems,  anyone  of  which 
shall  be  completely  capable  of  conveying  all  necessary 
data  to  the  decision  point. 

(5)  Two  data  processor  subsystem.^!,  either  of  which  shall 
be  completely  capable  of  pro..^:.  '■iing  all  required  data; 
Three  data  display  subsystema,  any  one  of  which  shall 
be  completely  capable  of  displaying  all  required  data. 

(6)  Six  independent  power  generating  devices,  any  four  of 
which,  when  operating  at  full  capacity,  shall  be  capa¬ 
ble  of  supplying  the  total  power  requirement.  In  normal 
operations,  five  generators  shall  be  on-line,  each 
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operating  at  80  percent  of  full  load  capacity.  Power 
lines  capable  of  transferring  power  with  no  more  than 
0.5  pe:  cent  power  loss  at  maximum  load. 

c.  Block  Diagraiis 

Two  functional  blocl  diagrams  representing  the  system  as  described 
during  the  definition  and  operational  phases  are  included  in  the  example. 

d.  Mission  Profile 

The  equipment  is  run  continuously  until  failure. 

3 .  Specification  of  Figur3(s)  of  Merit 

The  figure  of  merit  for  this  system  was  taken  to  be  "the  probability 
that  the  system  will  provide  a  30  mirute  warning,  given  an  enemy  air¬ 
borne  pre-emptive  attack  at  a  random  point  in  time." 

4.  Identifica^'ion  of  Accomtable  Factors 

The  example  includes  a  list  of  accountable  factors.  The  list  is  not 
necessarily  exhaustive  but  is  representative  of  those  requiring  considera¬ 
tion.  Some  of  the  example  factors  are  listed  below; 

a.  Atmospheric  phenomena 

b.  Enemy  actions  and  countermeasures 

c.  System  hardware 

d.  Maintenance  concept 

e.  Data  constraints 

f.  Data  sources. 

Publications  which  supply  generic  failure  rate  and  maintainability 
information  are  listed.  These  must  be  supplemented  by  contractors' 
bench  test  data. 

5.  Model  Construction 

a.  Delineation  of  Mission  Outcomes 

In  the  example,  consideration  is  given  to  the  possibility  that  a  warn¬ 
ing  of  an  attack  will  be  given  even  though  some  of  the  radar  equipments 
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are  in  a  failed  condition.  Therefore  an  evaluation  of  system  capabilities 
in  various  system  states  was  made. 

b.  Delineation  of  System  States 

Only  "significant"  system  states  were  considered  in  the  evaluation 
of  the  two  system  configurations  (1 -definition  phase,  and  Z-operational 
phase).  The  total  number  of  system  states  evaluated  was  reduced  to 
only  those  having  an  effect  on  system  capability.  These  were  called  "sig¬ 
nificant"  states.  A  further  simplification  was  made  by  treating  collec¬ 
tively  as  a  single  state  all  states  in  which  no  system  capability  exists. 

Diagrams  of  system  states  for  the  two  system  configurations  described 
are  included  in  the  example.  Four  (4)  system  states  were  considered  to  be 
significant  for  the  definition  phase  configuration  and  seventeen  (17)  for  the 
operational  phase  configuration. 

c.  System  Model 

The  basic  model  employed  for  both  system  configurations  is: 

E  =^'[^]  ^ 


where 

E  =  probability  that  the  system  will  provide  a  30 -minute  warning, 
given  an  enemy  airborne  pre-emptive  attack  at  any  random 
point  in  time; 

X  =  availability  vector;  A*  is  its  transpose 

=  probability  that  at  any  random  point  in  time,  the  system  will 
be  in  state  i,  where  i  can  be  any  integer  from  1  to  n, 
inclusive,  n  =  number  of  system  states  to  be  considered; 

[dJ  =  reliability  (dependability)  matrix 

=  probability  of  transition  from  system  state  i  to  system 
state  j  during  the  required  operating  period  (0.5  hours), 
given  state  i  at  the  beginning  of  this  period; 

=  capability  vector 

=  probability  that  the  system  can  successfully  perform  the 
required  functions,  given  that  the  system  is  in  state  j  during 
the  period  of  interest. 

Equations  are  developed  for  determination  of  the  components  of  A, 
^Dj  and  corresponding  to  the  various  system  states.  These  equations 
are  listed  for  the  system  configurations  considered. 


6.  Data  Acquisition 


Data  from  which  the  radar  equipments  failure  rates  and  repair  rates 
can  be  computed  are  identified. 

7.  Parameter  Estimation 

The  methodology  for  estimating  me  an -time -to -failure  (t^)  and  mean- 
repair -time  (t^)  is  discussed  in  the  example.  References  to  publications 
containing  prediction  models  for  estimating  mean-repciir-times  are  given. 

Again,  for  the  sake  of  brevity,  and  because  of  the  complexities 
involved  in  estimating  and  t^  "guesstimates"  are  made  of  t^  and  t^ 
nunb  ers  for  the  various  radar  equipments.  These  are  listed  in  a  table. 

8.  Model  Exercise 
a.  Availability 

The  availability  of  each  subsystem  is  determined  by  application  of 
and  t^  numbers  to  equation 
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These  results  are  then  used  to  determine  the  components  of  the 
availability  vector. 

b.  Dependability 

The  dependability  matrix  [d]  is  determined  for  the  four  system 
states  of  the  definition  phase  system  configuration  in  order  to  illustrate 
this  computation.  However  it  is  assumed  to  be  equal  to  one  (1)  for  the 
operational  system  configuration  because  of  the  relatively  short  duration 
of  the  mission  time.  This  useful  assumption  of  course  greatly  simplifies 
the  effectiveness  calculations. 
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c. 


Capability 


Numerical  values  for  each  component  of  the  capability  vector  were 
assumed  for  both  the  definition  phase  and  operational  phase  configurations. 
Each  assumed  value  represents  the  probability  of  successful  mission 
accomplishment  for  a  corresponding  system  state. 

d.  Effectiveness 

After  determination  of  the  availability  and  capability  vectors  and  the 
dependability  matrix,  the  effectiveness  (E)  is  determined  for  both  con¬ 
figurations  by  multiplication  of  these  three  terms.  Thus: 

E  =3:'[d]  c  . 
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APPENDIX  V 


SUMMARY  OF  THE  SPACECRAFT  SYSTEM  EXAMPLE 
(Complete  example  included  in  Volume  III.) 

This  example  illustrates  in  some  detail  a  method  by  which  Dependability 
of  a  spacecraft  may  be  determined  from  conservative  estimates  of  hardware 
reliability.  This  approach  is  suggested  in  the  absence  of  large  amounts  of 
test  data  on  the  vehicle  being  evaluated.  This  usually  occurs  during  the 
program  definition  and  early  system  acquisition  phases  of  programs  on  sub¬ 
stantially  new  systems.  It  is  also  useful  for  evaluating  extremely  costly 
systems  of  which  only  a  few  are  to  be  constructed.  No  effort  will  be  made 
in  this  example  to  treat  Availability  or  Capability,  beyond  illustrating  their 
tie-in  with  Dependability  to  calculate  Effectiveness;  i.e., 

E  =  A"  [d]  U  . 

The  assumed  purpose  of  this  evaluation  is  the  determination  of  criti¬ 
cal  elements  in  the  proposed  spacecraft  configuration. 

1.  Mission  Definition 

The  spacecraft  system  shall  be  capable  of  placing  a  variety  of  payloads 
including  multiple  satellites  into  precise  orbits  about  the  earth.  It  shall 
have  the  capability  of  restarting  in  space  after  a  sufficient  coast  v.  o3*iod 
dependei.'.t  on  the  specific  payload  and  attitude  orientation  in  space.  The 
system  shcill  be  designed  as  an  upper  stage  rocket  propulsion  vehicle. 

2.  System  Description 

The  system  described  herein  is  a  spacecraft  for  placing'  satellites  in 
earth  orbits.  The  spacecraft  is  a  liquid-propellcint  upper-stage  rocket 
propulsion  vehicle  providing  all  the  control  elements  necessary  for  placing 
a  variety  of  payloads  in  precise  orbits  about ^the  earth. 

The  spacecraft  is  composed  of  four  major  subsystems,  namely,  pro¬ 
pulsion,  forward  section,  ordnance  (except  range  safety)  and  structure. 
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Further  analysis  of  each  subsystem  and  its  function  is  made  in  the  example. 
Included  are  block  diagrams,  mission  sequences,  environments,  etc. . 

3.  Specification  of  Figure  of  Merit 

The  system  effectiveness  is  evaluated  in  terms  of  the  probability  that 
the  system  will  achieve  a  given  mission.  This  example  concentrates  on 
the  determination  of  the  dependability  matrix  in  the  effectiveness  model 

E  =  s'  ''d]  Z 

where 

A  is  a  1  X  3  row  vector  comprising  the  relative  levels  of,  or 

probabilities  of  availability  in,  each  of  three  states  of  readiness 
which  are  not,  themselves,  calculated  in  this  example 

[pj  is  a  3  X  3  dependability  matrix  comprising  transition  proba¬ 
bilities  of  performing  (i.  e.  ,  completing  a  mission)  in  each  of 
three  system  states  depending  on  the  initial  state,  or  readiness 

^  is  a  3  X  1  column  vector  of  the  relative  payoffs,  or  success 

probabilities,  associated  with  each  state  of  system  performance  -- 
again,  not  calculated  for  this  example. 

4.  Identification  of  Accountable  Factors 


The  following  items  are  quantitatively  considered  in  evaluating  the 
system  effectiveness  for  the  upper -stage  spacecraft. 


a. 

The  mean  time  to  failure 

b. 

The  mission  length 

c. 

The  structural  integrity 

d. 

The  maintenance  policy 

e. 

The  system  capability. 

Model  Construction 

a. 

Delineation  of  Mission  Outcomei 

There  are  three  possible  mission  outcomes: 

(1)  perfect  operation  --in  desired  orbit  with  all  data 

(2)  imperfect,  but  acceptable  operation  --  in  orbit,  partial  data 

(3)  unacceptable  operation  --no  orbit,  no  data. 
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b.  Delineation  of  System  States 

Three  states  are  defined  in  accordance  with  the  specified  mission 
outcomes: 


State 

Designation 


Equipment 

Condition 


1  perfect  operation 

2  imperfect,  but  acceptable  operation 

3  unacceptable  operation 

c.  Models 


Jn  the  example,  basic  reliability  models  from  which  the  dependability 
matrix  is  developed,  are  described. 


Consideration  is  given  to  the  probability  that  all  parts  will  function 
properly  and  to  the  probability  that  all  essential  parts  will  function 
properly.  Both  time -stress  and  peak-stress  analyses  are  demonstrated, 
as  are  methods  for  adjusting  for  environmental  and  application  conditions. 
Further,  an  analysis  of  the  reliability  of  structural  components  is  made. 

These  analyses  are  employed  in  developing  the  dependability  matrix 
of  the  following  form: 


'^ll  ^12  ^^13 


0  0  1 


where 

dll  =  probability  that  spacecraft  will  have  no  failure  during  mission; 
given  that  it  is  initially  non -failed 

di2  =  probability  that  the  spacecraft  will  have  one  or  more  non- 

critical  failures  in  the  mission,  given  that  it  was  initially  non- 
failed 

di2  -  probability  that  the  spacecraft  will  have  one  or  more  critical 
failures  in  the  mission;  given  that  it  was  initially  non-failed 

^22  ~  probability  that  the  spacecraft  will  have  a  critical  failure  in  the 
mission;  given  that  it  has  initially  one  or  more  non-critical 
failures 
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probability  that  the  spacecraft  will*  have  one  or  more 
critical  failures  in  the  mission;  given  that  it  initially  has 
one  or  more  non -critical  failures. 

6.  Data  Acquisition 

(See  parameter  estimation.) 

7.  Parameter  Estimation 

The  subsystem  failure  rates  shownin  the  example  for  the  variovis  mis¬ 
sion  segments,  are  derived  fror '  'c  component  failure  rates  which  are 
reciprocals  of  component  mean- tin-'  ‘‘o-failure  as  estimated  from  industry 
experience  and  testing. 

Component  failure  rates  are  adjusted  for  environmental  stress  and  for 
condition  of  use  or  application,  then  combined  into  assembly  and  subsystem 
failure  rates.  Since  component  data  were  obtained  from  various  industry 
and  test  sources  the  adjustment  factors  included  escalation  factors  from 
one  source,  such  as  generic  industry  lists  to  another  source,  such  as  hangar 
check-out  data.  The  joint  effects  of  all  these  adjustments  are  reflefcted  in 
K-factors  by  which  failure  rates  from  the  various  sources  are  multiplied 
to  obtain  rates  applicable  to  various  aspects  of  the  mission. 

Since  the  failure  rates  are  applicable  to  specific  aspects  of  the  mis¬ 
sion,  various  segments  of  the  mission  are  treated  individually. 

Consideration  is  also  given  to  those  failures  which  are  not  random 
with  respect  to  time.  This  occurs  when  peak  stress  during  mission  opera¬ 
tion  exceeds  the  strength  of  the  part.  Probability  statements  that  this 
kind  of  failure  will  not  happen  may  be  termed  peak  stress  reliability  and 
are  included  in  the  example. 

Included  (without  specific  modeling  or  calculation)  in  the  capability 
vector  are  the  concepts  of  expected  return  (cr  payoff)  from  perfect, 
acceptable  or  unacceptable  payload  injection  into  orbit,  respectivijly, 
weighted  for  the  reliability  of  the  payloads  and  of  the  externally  furnished 
command  guidance  subsystem. 
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8.  Model  Exercise 


The  dependability  matrix 


0 

0 


is  calculated  in  this  example  to  be 


0 

0 


where  the  subscripts  refer  to  the  functional  states 

1  Perfect 

2  Imperfect  but  acceptable 

3  Unacceptable, 

and  the  double  subscript,  ij,  denotes  the  probability  of  entering  state  j 
when  the  initial  state  of  readiness  is  state  i. 


Calculations  reflecting  assumed  values  for  Availability  and  Capability 
are  included  in  the  example. 
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